PixelPointingBenchmark – Simple tests reveal surprising gaps

Show HN: PixelPointingBenchmark – Simple tests reveal surprising gaps

Based, PixelPointingBenchmark – Simple tests reveal surprising gaps works in show hn launch.

Profile

We built a small open-source benchmark to test how well vision-enabled LLMs handle pixel-level pointing on screens. Instead of complex UI screenshots, we use synthetic images with basic shapes and clean backgrounds to isolate spatial reasoning and coordinate accuracy.The results were surprising:Many top models miss by tens to hundreds of pixels on trivial tasks (e.g., center of a purple circle or red square). High run-to-run variance in some models (different answers on the same image/prompt). Performance flips dramatically with resolution or aspect ratio changes. Claude Sonnet and Claude Haiku are consistently near-perfect (0–1px error), while others show clear gaps. We wrote a detailed blo

The company sits in the show hn launch segment, where it competes on its core product or service.

It is recorded at the Show HN launch stage.

Website: autodevice.github.io/PixelPointingBenchmark