SchemaGate’s new data-lookups improve BIRD—but its Spider result is the warning worth reading

SchemaGate 1.13 adds a useful answer to a familiar NL2SQL failure: sometimes the schema is not enough. A user asks for customers in “Prague,” while the database stores Praha; or asks for a closed status while the field contains C. The new deep_answer MCP tool can run a small exploratory read, show the model up to 10 rows, request more permitted schema, and only then produce final SQL. SchemaGate’s documentation says every intermediate query still passes through the same read-only, caller-scope and audit checks as an ordinary query.

The default is selective, not fully agentic

The release’s default adaptive mode first writes a normal query and can repair it twice. It enters the multi-step loop only if the query still errors, returns no rows, or returns only zero values. Operators must explicitly enable the feature because exploratory rows are sent to the configured model provider; it is off by default. Exploration queries have a 10-second default limit, the loop defaults to six model calls and is capped at 12, and cached exploration results are isolated by caller, role, catalog version and SQL. The v1.13 release also adds role- and table-scoped business rules plus per-statement timeouts for PostgreSQL, Oracle, MySQL and SQLite.

The benchmark gain is real—and narrow

On 220 BIRD development questions across 11 SQLite databases, using Gemini 2.5 Flash, plain generation scored 36.8% execution accuracy. Always-deep mode reached 54.1%, while adaptive mode reached 54.5% at 1.68 model calls per question. The project reports a paired exact-test result below 0.0001 and attributes about 90% of the wins to exploratory steps that exposed real values.

The same documentation also gives reasons not to generalize that result. Deep mode scored 39.5% versus 22.3% plain on a 215-question PostgreSQL setup and 48.6% versus 32.7% on Oracle, but it did worse than plain on three of the 11 Oracle databases. On 135 Spider 2.0-lite SQLite tasks with business rules, adaptive scored 42.2% against 45.2% for plain generation plus repairs; the project calls that a statistical tie with p=0.48.

What teams should test

The practical lesson is not “always let the model inspect rows.” It is to route exploration only to failures that plausibly come from hidden value conventions, then measure by database and workload. SchemaGate’s own results show why aggregate gains need per-database inspection. Teams should also treat sending sample rows to a model provider as a separate data-governance decision, even when SQL authorization remains intact.

Version 1.13 is therefore notable for two things: it ships a constrained, auditable way to inspect data before answering, and it publishes the negative benchmark result that defines where the technique may not help.