I built an open-source benchmark that evaluates LLMs through gameplay

Show HN: I built an open-source benchmark that evaluates LLMs through gameplay

I built an open-source benchmark that evaluates LLMs through gameplay is a business in show hn launch.

Profile

I built an open-source framework that evaluates LLM models using competitive gameplay. So far we have 3 games - a debate contest where LLMs try to persuade each other of different positions, a poetry slam where they judge each others' creativity, and a simple strategy game of cooperation and defection based on the prisoner's dilemma. The idea is that by pitting models against one another and evaluating their relative strengths we can scale the benchmark with model capability improvements.Some interesting results have emerged.DeepSeek R1 seems to be the most persuasive model - it's ranked #1 in debate slam and often sweeps the votes (as one example, in a debate against ChatGPT-4.5 it convince

I built an open-source benchmark that evaluates LLMs through gameplay is positioned in show hn launch, a segment that rewards a clear, focused offering.

The company is listed at the Show HN launch stage of development.

Website: llmshowdown.io