The strongest signal from three new data-agent tools is what their authors admit does not work

Three open-source data-agent projects created in the past two days are competing on different layers of the stack, but their most useful shared feature is methodological: each makes a boundary visible instead of hiding it behind a fluent demo. Keygraph maps schema context before a model writes SQL, InsightPilot measures the generated-query loop, and incognito-sql tries to keep personal values away from the model after a query is written.

Context before generation

Keygraph reads database metadata and emits join paths, keys, table grain, status codes and business context through MCP, JSON, Markdown and an offline explorer. Its repository says it supports MySQL/MariaDB, PostgreSQL, SQLite, SQL Server, ClickHouse, MongoDB and Metabase, exposes 15 MCP tools, and makes no LLM calls. Just as importantly, inferred relations carry confidence and a reason rather than being presented as declared truth. The project is MIT-licensed but only had two GitHub stars when checked on Nov. 4, so this is an architectural entrant, not adoption evidence. Source

That separation matters because a text-to-SQL model can produce executable SQL while choosing the wrong join or grain. Keygraph does not claim to solve generation or execute an agent’s query; its documented loop asks the agent to inspect candidate tables, retrieve a path, examine indexes and verify low-confidence relations before using its own read-only database connection. Source

An ablation that declines to flatter the architecture

InsightPilot reports a two-run, 60-question evaluation on a small SQLite snapshot. Its full six-agent configuration reached 50.0% strict execution accuracy; removing the critic reached 50.8%, while removing schema retrieval or using a single chain reached 45.0%. The authors explicitly say the 0.8-point critic difference is noise and that two runs cannot establish a causal gain from the five-point schema-retrieval difference. They also disclose that eight contradictory gold questions were corrected and that part of the improvement from an earlier version came from fixing the evaluation set, not the system. Source

Those numbers are not leaderboard-ready: the data covers only two dates, the deployment is single-machine, and there is no row-level security or masking. But a committed negative result is more decision-useful than a multi-agent diagram with no ablation. Source

Privacy claims need an audit, not applause

Incognito-sql takes the opposite boundary. It parses agent-written SQL, traces outputs to protected base columns, regenerates a read-only query and returns stable tokens in place of declared identifiers. Its vendor-authored benchmark reports zero hidden-value disclosures across 2,894 attacks for both strict and permissive profiles, with 164 of 180 and 180 of 180 honest tasks answered respectively. The same README labels the software an unaudited MVP and warns that permissive mode can reveal order, length, prefixes and containment information within configured budgets. Source

The practical lesson is not that these three young repositories are production-ready. It is that open-source data agents are beginning to compete with inspectable contracts: which context is deterministic, which module failed to improve accuracy, and which privacy leakage remains intentionally permitted. Those are claims an evaluator can falsify. That is a healthier foundation than another perfect-looking chat transcript.