Making Postgres Self-Describing for LLMs with a Semantic Catalog

Show HN: An Open-Source Eval Suite That Helps You Fix Postgres-Based Text-to-SQL

Based, Making Postgres Self-Describing for LLMs with a Semantic Catalog works in show hn launch.

Profile

We've been building text-to-SQL at TigerData and kept hitting the same problem: evaluation tools that tell you your accuracy score but nothing about how to improve it.Getting a 60% pass rate is meaningless if you don't know whether failures are from bad schema retrieval or poor SQL generation. It's the difference between actionable insights and meaningless benchmarketing.So we built, and are now open-sourcing, text-to-sql-eval with a simple insight: run every query three different ways:- Normal mode - let the system retrieve schema and generate SQL - Full schema mode - provide all tables to test upper bound accuracy - Golden tables mode - give it the right tables to isolate reasoning issuesT

As a business in show hn launch, Making Postgres Self-Describing for LLMs with a Semantic Catalog is listed here with the public details available about it.

The company is listed at the Show HN launch stage of development.

Website: www.tigerdata.com/blog/text-to-sql-eval-open-source