Open-source study to measure end user satisfaction levels with LLMs

Show HN: Open-source study to measure end user satisfaction levels with LLMs

Open-source study to measure end user satisfaction levels with LLMs operates in show hn launch.

Profile

The LLM challenge - an online study - aims to answer a simple question: what is the quality corridor that matters to end users when interacting with LLMs? At what point do users stop seeing a quality difference and at what point do users get frustrated by poor LLM quality.The project is an Apache 2.0 licensed open source project available on Github: https://github.com/open-llm-initiative/llm-challenge. And the challenge is hosted on AWS as a single-page web app, where users see greeting text, followed by a randomly selected prompt and a LLM response, which they must rate on a likert scale of 1-5 (or yes/no rating) that matches the task represented in the prompt.The study uses pre-generated p

Open-source study to measure end user satisfaction levels with LLMs is positioned in show hn launch, a segment that rewards a clear, focused offering.

The company is listed at the Show HN launch stage of development.

Website: open-llm-initiative.com/challenge