A tool to benchmark LLM APIs (OpenAI, Claude, local/self-hosted)

Show HN: A tool to benchmark LLM APIs (OpenAI, Claude, local/self-hosted)

A tool to benchmark LLM APIs (OpenAI, Claude, local/self-hosted) is a company in show hn launch.

Profile

I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp).It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluencyDemo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-testThe goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including the growing number of third-party “proxy” or “cheap LLM API” services.It supports: • OpenAI-compatible APIs (official + proxies) • Claude (via An

The company sits in the show hn launch segment, where it competes on its core product or service.

It is recorded at the Show HN launch stage.

Website: llmapitest.com