live wire
▸Oracle Select AI generates SQL/PGQ for property graphs—but errors if one AI profile mixes graphs with tables or viewsOracle▸Domo’s data team standardizes MCP builds with shared scaffolding; beta toolkits inherit user permissionsDomo▸Databricks proposes five outcome signals for agentic BI—and says dashboard counts and license usage miss decision qualityDatabricks▸One vibe-coded database app needed 56+ human test hours and three repair rounds despite AI Studio’s verification claimsMancas & Mancas / arXiv▸BigQuery AI.IF’s default cost mode trains a local model for most rows—and drops row-error thresholdsGoogle Cloud▸AWS’s multi-tenant data-agent pattern gives the broker Lambda no data permissions; Cognito roles carry Lake Formation scopeAWS Big Data Blog▸Amazon OpenSearch’s optimized engine adds nested agent traces, but AWS reserves it for log analytics—not vector searchAWS▸Two interpretation-based provenance semantics cannot express SQL bag difference, new theorem showsKolaitis / arXiv▸Databricks Feature Views reuse one definition for training and live scoring, but online lookup still returns latest available stateDatabricks▸Databricks Genie’s Slack and Teams previews default channel answers to public, even when viewers lack source-data accessDatabricks▸Snowflake’s two-tool GraphRAG agent beat a seven-tool graph setup 70% to 60% on 22 ontology questionsSnowflake Engineering▸Semantix opens an Apache-2.0 semantic-first data-agent stack, but labels the project WIP and has no release or adoption yetGitHub / icosmos-space▸Confluent defines Streamhouse as shared current state for agents and apps, but says broader semantics and governance are still to comeConfluent▸Graph-repair retrieval cut F1 by 0.126 on one suite; a learned router used fewer tokens than always-on retrieval in all sixLin et al. / arXiv▸Oracle Select AI generates SQL/PGQ for property graphs—but errors if one AI profile mixes graphs with tables or viewsOracle▸Domo’s data team standardizes MCP builds with shared scaffolding; beta toolkits inherit user permissionsDomo▸Databricks proposes five outcome signals for agentic BI—and says dashboard counts and license usage miss decision qualityDatabricks▸One vibe-coded database app needed 56+ human test hours and three repair rounds despite AI Studio’s verification claimsMancas & Mancas / arXiv▸BigQuery AI.IF’s default cost mode trains a local model for most rows—and drops row-error thresholdsGoogle Cloud▸AWS’s multi-tenant data-agent pattern gives the broker Lambda no data permissions; Cognito roles carry Lake Formation scopeAWS Big Data Blog▸Amazon OpenSearch’s optimized engine adds nested agent traces, but AWS reserves it for log analytics—not vector searchAWS▸Two interpretation-based provenance semantics cannot express SQL bag difference, new theorem showsKolaitis / arXiv▸Databricks Feature Views reuse one definition for training and live scoring, but online lookup still returns latest available stateDatabricks▸Databricks Genie’s Slack and Teams previews default channel answers to public, even when viewers lack source-data accessDatabricks▸Snowflake’s two-tool GraphRAG agent beat a seven-tool graph setup 70% to 60% on 22 ontology questionsSnowflake Engineering▸Semantix opens an Apache-2.0 semantic-first data-agent stack, but labels the project WIP and has no release or adoption yetGitHub / icosmos-space▸Confluent defines Streamhouse as shared current state for agents and apps, but says broader semantics and governance are still to comeConfluent▸Graph-repair retrieval cut F1 by 0.126 on one suite; a learned router used fewer tokens than always-on retrieval in all sixLin et al. / arXiv
nl2sql.ai

research · since 2026-10-22

Semantic Layer × Harness Bench

PCalvanelli

Configuration benchmark comparing raw SQL, glossary-assisted SQL, dbt MetricFlow and Malloy interfaces with and without an agent harness over one frozen analytics warehouse.

website ↗repository ↗
benchmarksemantic-layermetricflowmalloytext-to-sql
desk notes

No declared repository license as of 2026-10-28. Current author-reported Sonnet 5.5 results use 22 prompts: MetricFlow 21/22 across three runs, Malloy 21/22 across three runs, raw SQL 12/22 in one run. Repository warns the sample is small (about ±18 percentage points with 22 prompts) and the published runs isolate only interface effects, not the full model/interface/harness matrix.