Analytics agents need typed failures at every layer—not another generic retry loop

The most important analytics-agent feature shipping this week is not a better chat box. It is a clearer failure contract.

Three changes at different layers point in the same direction. DataTalk’s NL2SQL prototype measures when semantic validation rescues or damages a generated query. Databricks Lakeflow Designer gives data-quality checks explicit pass, block and fail paths. Cube 1.8.14 turns malformed successful-looking responses into typed request errors and makes distributed cancellation handles unique. Together, they show why an autonomous query loop cannot safely operate on a single binary signal: success or failure.

Generation failures need evidence, not just rejection

DataTalk reports 84 exact-result matches across 102 questions with semantic validation and repair enabled. Its report displays that count as 82.3%, while the exact fraction rounds to 82.4% to one decimal; this analysis uses the underlying count to avoid that presentation mismatch. The no-semantic-validation ablation reports 74.5% while static SQL checks remained. But first-attempt accuracy moved the other way: 66.7% with semantic checks and 71.6% without them. The project says some already-correct queries were rejected and then damaged during repair. That is author-reported evidence from one 3B model and one job-listings database, not an independent benchmark, but the ablation identifies the right operational metric: count false interventions as well as rescued failures. DataTalk’s evaluation report documents the comparison.

Pipeline failures need policy

Databricks’ current Lakeflow Designer operator reference exposes three Guardrails outcomes: warn and continue with passed rows, warn and block the passed output while retaining failed rows and error details, or fail the workflow before downstream operators run. The same reference gives Filter separate Included and Excluded outputs. Those controls do not decide policy for the operator; they make the policy executable. The team still has to declare which violations are informational, quarantinable or fatal. Databricks documents those branches in its built-in operator reference.

Runtime failures need types

Cube’s 1.8.14 client now checks whether an HTTP 200 response contains the expected load-response shape. If not, it raises a RequestError instead of allowing a raw TypeError or unhandled rejection in callback-style paths. A separate fix makes cancellation handles unique across nodes after maintainers identified a case where matching handles could cancel unrelated queued work. The release notes link both fixes.

The pattern is practical. An agent deciding what to do next must distinguish a rejected SQL candidate, quarantined source rows, a malformed upstream response, a cancelled branch and an empty valid result. Treating all five as “retry” can amplify load, hide data-quality defects or rewrite a query that was already correct.

The rollout test should therefore cover the whole failure chain. Measure validator rescues and false interventions; assign an owner and action to every pipeline branch; inject malformed 200 responses; and cancel concurrent work while checking that unrelated queries complete. Autonomy becomes safer when every layer supplies a reason the orchestrator can act on—not when the model is simply allowed to try again.