Snowflake’s ontology test favors fewer agent tools—but the benchmark is deliberately narrow

Snowflake has published a useful counterexample to the idea that a more capable data agent necessarily needs more tools. In a controlled biomedical evaluation, its two-tool GraphRAG configuration scored 70.0% success, while a seven-tool knowledge-graph configuration scored 60.0%. A semantic-view baseline scored 50.0%, and GraphRAG augmented with curated terminology mappings reached 78.2%.[^1]

The result is worth reading as an architecture experiment, not a product leaderboard. Snowflake created 22 ontology-dependent questions and ran each agent five times. The dataset combined a cell ontology containing 33,651 terms and roughly 50,000 edges with PRISM drug-screening data covering 4,518 drugs, 578 cancer cell lines and more than 2.6 million viability measurements.[^1]

Why the smaller agent won

The seven-tool knowledge-graph agent could traverse hierarchy tables through recursive SQL and call specialized stored procedures. That gave it deterministic reach across deep taxonomies, but it also expanded the orchestration problem: the agent had to select and sequence the right tools, and procedures required exact concept names.[^1]

Snowflake’s GraphRAG design moved more work into preprocessing. It flattened each ontology concept into a searchable profile containing names, definitions, synonyms, neighborhood information and attributes aggregated from descendants. At runtime, the agent used one search tool and one Cortex Analyst SQL tool. Snowflake says this reduced the decision surface and produced lower run-to-run variance than the other tested configurations.[^1]

That distinction matters for production NL2SQL systems. If a query depends on hierarchical meaning—“all electronic components,” for example—the operating choice is not simply graph database versus vector search. Teams can choose between runtime traversal, which preserves flexible relationship reasoning, and a precomputed index, which turns much of that reasoning into data maintenance and retrieval.

The 78.2% ceiling came with maintenance

The best score did not come from GraphRAG alone. Snowflake added eight hand-curated cell-type-to-tissue mappings and definitions for composite terms to the agent instructions. That lifted success from 70.0% to 78.2%, but Snowflake also identifies the trade-off: the mappings are static, manually maintained and only cover concepts anticipated by the designers.[^1]

The practical pattern is therefore layered. Keep the relational semantic view as the governed SQL surface; precompute broad ontology context where retrieval can replace repeated traversal; reserve a small, reviewed mapping layer for domain phrases that hierarchy and synonym search cannot resolve.

What the benchmark does not establish

Snowflake explicitly calls the evidence directional. The test uses one simplified biomedical problem, one ontology, 22 questions and vendor-tuned prompts and tool descriptions. It does not test multiple ontologies, provenance and uncertainty across mappings, workload drift, index-refresh failures or independent reproduction.[^1]

The defensible takeaway is narrower than “GraphRAG beats knowledge graphs.” In this setup, relocating structural reasoning into maintained context reduced runtime orchestration and improved measured consistency. For data-agent teams, that is a concrete reason to benchmark tool count and context construction together—rather than treating every additional tool as free capability.

[^1]: Snowflake Engineering, “Ontology-grounded Reasoning with Cortex Agents”