live wire
IBM makes watsonx Orchestrate AgentOps, custom LLM judging and Bedrock-agent discovery generally availableIBMSchemaGate 0.1.45 fixes broken Oracle ADB wallet connections and an OCI stack pinned 28 releases behindSchemaGatePDI’s Amazon Quick procurement agent grounds spend answers in vendor, category and contract contextAWS Business Intelligence BlogBigQuery’s ML.METRICS example returns 0.84 accuracy but 0.30 macro-F1 on the same 100-row classification queryGoogle Cloud BigQuery docsSchemaGate 0.1.44 auto-selects sentence embeddings, lifting bundled-schema retrieval from 90/98 to 93/98SchemaGateSchemaGate 0.1.43 adds read-only SQL execution with per-principal table checks—and documents unauthenticated client assertionsSchemaGateDatabox adds reusable AI Analyst Skills with personal/company scope, auto-matching and marketplace installsDataboxFabric previews an AI builder for data-agent instructions, source guidance and example queriesMicrosoft FabricDatabricks trains data-agent retriever to stop early or spend bounded extra search steps, reporting 5.8-second latencyDatabricksThoughtSpot adds SpotterCode coding agent to its Visual Embed PlaygroundThoughtSpotLongMemEval-S audit: 67–73% of restore-fixable 80k-budget errors came from evicted evidence under three policiesarXivSchemaGate 0.1.42 adds dimension-aware retrieval and fixes complex multi-table SQL promptsSchemaGateSnowflake agent toolsets can silently drop inherited tools when callers lack accessSnowflake DocumentationLooker’s VS Code extension reaches GA with MCP-assisted LookML generation, editing and validationGoogle Cloud Looker release docsIBM makes watsonx Orchestrate AgentOps, custom LLM judging and Bedrock-agent discovery generally availableIBMSchemaGate 0.1.45 fixes broken Oracle ADB wallet connections and an OCI stack pinned 28 releases behindSchemaGatePDI’s Amazon Quick procurement agent grounds spend answers in vendor, category and contract contextAWS Business Intelligence BlogBigQuery’s ML.METRICS example returns 0.84 accuracy but 0.30 macro-F1 on the same 100-row classification queryGoogle Cloud BigQuery docsSchemaGate 0.1.44 auto-selects sentence embeddings, lifting bundled-schema retrieval from 90/98 to 93/98SchemaGateSchemaGate 0.1.43 adds read-only SQL execution with per-principal table checks—and documents unauthenticated client assertionsSchemaGateDatabox adds reusable AI Analyst Skills with personal/company scope, auto-matching and marketplace installsDataboxFabric previews an AI builder for data-agent instructions, source guidance and example queriesMicrosoft FabricDatabricks trains data-agent retriever to stop early or spend bounded extra search steps, reporting 5.8-second latencyDatabricksThoughtSpot adds SpotterCode coding agent to its Visual Embed PlaygroundThoughtSpotLongMemEval-S audit: 67–73% of restore-fixable 80k-budget errors came from evicted evidence under three policiesarXivSchemaGate 0.1.42 adds dimension-aware retrieval and fixes complex multi-table SQL promptsSchemaGateSnowflake agent toolsets can silently drop inherited tools when callers lack accessSnowflake DocumentationLooker’s VS Code extension reaches GA with MCP-assisted LookML generation, editing and validationGoogle Cloud Looker release docs
nl2sql.ai
benchmarkBenchmark audit

Cortex AISQL’s 8× cost result is a model of a system—not a measured system result

The paper combines measured component results into a 7.8× realistic estimate, but it has not yet run the composed optimizer end to end at that scale.

Chart of modeled Cortex AISQL cost reduction, with 7.8× as an estimate.
AI-generated illustration
By The Benchmark Desk· Aug 28, 2026the quick take — two AI hosts go live when you do

A new Cortex AISQL paper offers a useful cost model—and an easy number to overread. Its headline result is a roughly 8× total-cost reduction from combining online filter ordering with proxy-to-oracle cascade routing. But the paper’s own evaluation makes the status precise: 7.8× is an analytical estimate for a representative workload, not an end-to-end measurement of the two components running together at production scale. (paper, Sections 6.3–6.4)

What the calculation assumes

The modeled workload contains 1 million rows and a conjunction of five semantic predicates. The calculation assumes filter ordering cuts expected predicate evaluations from five to 1.25 per row, while the cascade sends 30% of evaluated predicates to an oracle model and charges the proxy at 5% of oracle cost. Under independence, those factors reduce modeled cost to 8.75% of baseline, an 11.4× upper bound. (paper, Section 6.3)

Those inputs are anchored in separate component results. The Larch-Sel ordering method reports typical 4×–8× token-overhead reductions on a mixed AND/OR workload, with a 19× ceiling against Palimpzest and Quest. GAMCAL reaches F1 ≥ 0.95 on six classification, filtering and join benchmarks, requiring 17%–68% oracle delegation; it uses up to 58% fewer oracle calls than LOTUS’s single-pass cascade and leads on five of six benchmarks. Those are measured component-level findings, not a measured composed run. (paper, Sections 4–5)

Why the estimate falls to 7.8×

The paper then models two steady-state interactions. Rows surviving earlier predicates can be harder for the cascade, increasing delegation; proxy decisions can also inject noisier labels back into the filter-ordering learner, weakening short-circuit selection. At midpoint penalties of 1.25× and 1.2×, respectively, the estimate falls from 11.4× to about 7.8×. Its pessimistic corner is 5.7×; the optimistic corner returns to 11.4×. A third cold-start penalty is omitted from the long-run figure because the paper estimates it affects roughly the first 1% of batches for the million-row setup. (paper, Section 6.4)

The distinction matters because “production case study” describes Cortex AISQL as the substrate, while the combined number remains a model assembled from earlier component evidence. The paper explicitly names empirical validation of the composition argument at scale as future work. (paper, Sections 3 and 9)

For practitioners, 7.8× is therefore a planning hypothesis with declared parameters—not a benchmark result to copy into a capacity forecast. A defensible reproduction should report the actual row count, predicate count, proxy/oracle cost ratio, delegation rate, quality target, and end-to-end cost with both learners enabled. Until that experiment exists, the paper’s most valuable contribution is not the multiplier; it is the accounting that shows exactly where the multiplier can shrink.

Filed by The Benchmark Desk. Corrections: desk@nl2sql.ai · Our standards →

comments · 0

    Comments are moderated before they appear. Your email is used once to confirm it is you — never shown, never sold. Corrections and questions get an answer from the desk when we have one.