The newest NL2SQL projects are publishing their limits before they find users

A curious pattern is showing up at the smallest edge of the NL2SQL ecosystem: projects with almost no visible adoption are publishing unusually specific evidence about where they fail.

This is not a claim that a few repositories represent the whole market. It is a useful launch signal. Five projects created or newly surfaced in early September—AgentCrew, Datatide, Jeeves, Metric Agent and TableTalk—each had no more than two GitHub stars when checked on September 10. Yet their documentation leads with controls and caveats that older “chat with your database” demos often postponed.

The negative result is moving into the README

TableTalk reports 67% execution accuracy on a fixed 100-question Spider sample using a local Qwen2.5 7B model. It also leaves one behavior test visibly failing: an ambiguous “top customer” question makes the model assume revenue instead of asking for clarification. Its README describes an earlier prompt-injection failure, the data-header and tool-budget mitigations added afterward, and the remaining limits of its Python sandbox.

AgentCrew is blunter still. Its published evaluation says a multi-stage agent used roughly four times the tokens and three times the latency of a direct baseline without a measurable accuracy advantage on its mostly single-query workload. The project argues that orchestration bought bounded failure behavior and traceability rather than better accuracy—and says a valid remeasurement now needs held-out questions.

Jeeves publishes two benchmark ranges on uncurated schemas and summarizes them for operators as roughly one wrong answer in four. It also calls out a non-monotonic result by join width rather than smoothing it into a single score.

Controls are becoming product surface

The projects also expose their boundaries as user-facing design choices. Datatide documents dataset-level authorization, a single-statement SELECT/WITH guard, read-only PostgreSQL attachment and row/time limits—but explicitly says its guard is lexical rather than a full AST parser and that row- and column-level controls are not yet available.

Metric Agent takes a different route: retrieve a published metric definition first, constrain SQL with that semantic layer, and ask the user to choose a subject when retrieval misses instead of forcing a runnable query. TableTalk and AgentCrew both pair parser-level SQL checks with physically read-only SQLite connections and explicit execution budgets.

None of this proves production readiness. The repositories are new, visible adoption is negligible, and several support only narrow database or deployment paths. The meaningful shift is editorial: limits, failed tests and threat boundaries are no longer buried after the quick-start. For evaluators, that makes a tiny project more legible than a polished demo with no denominator. For maintainers, it raises the launch bar: showing the failure is becoming part of shipping the tool.