New eval from SWE-bench team evalutes LMs based on goals not tickets

Show HN: New eval from SWE-bench team evalutes LMs based on goals not tickets

New eval from SWE-bench team evalutes LMs based on goals not tickets is a company in show hn launch.

Profile

Current evals test LMs on tasks: "fix this bug," "write a test"But we code to achieve goals: maximize revenue, cut costs, win usersMeet CodeClash: LMs compete via their codebases across multi-round tournaments to achieve high-level goals.Because real software dev isn’t about following instructions. It’s about achieving outcomes.Here's how it works:Two LMs enter a tournament. Each maintains its own codebase.Every round:1. Edit Phase: LMs modify their codebases however they like 2. Competition phase: Codebases battle in an arena. 3. RepeatThe LM that wins the majority of rounds is declared winner.Arenas can be anything like games, trading sims, cybersec envs. We currently have 6 arenas impleme

New eval from SWE-bench team evalutes LMs based on goals not tickets is positioned in show hn launch, a segment that rewards a clear, focused offering.

Its current recorded stage is Show HN launch.

Website: codeclash.ai