Word Game Bench – evaluating language models on word puzzles

Show HN: Word Game Bench – evaluating language models on word puzzles

Word Game Bench – evaluating language models on word puzzles is a business in show hn launch.

Profile

Hey HN!Word Game Bench is a fun benchmark for evaluating language models on word puzzle games. It is a relatively hard benchmark, where no model currently scores above 50% average win rate.Currently, the models are evaluated on 2 tasks:1. Wordle is a word puzzle game where the player has to guess a 5-letter word in 6 attempts. For each letter in the guessed word, the player receives feedback on whether the letter is in the target word and in the correct position.2. Connections is a word association game where the player has to group 16 words into 4 categories of 4 words each. The player doesn't know the categories beforehand, and has to group the shuffled words based on their associationsI b

The company sits in the show hn launch segment, where it competes on its core product or service.

Its current recorded stage is Show HN launch.

Website: wordgamebench.github.io