BigQuery’s correlation chat needs a function-selection benchmark—not a fluent answer

Google added ML.CORRELATION to BigQuery conversational analytics on September 10, 2026. The function is in preview, and it can be invoked from chats with data agents or data sources as well as from verified SQL queries. That is a meaningful expansion: a natural-language question can now route into a statistical table-valued function rather than a simple aggregation. It is not, however, an accuracy result. Google’s release note and function reference publish no benchmark showing how often the conversational layer selects the intended statistical setup.

The hard part is choosing the contract

The SQL function itself has explicit knobs. It takes one numerical target column, one or more numerical correlation columns, optional dimensions and a method. Google supports Pearson, Spearman and Kendall, with Pearson as the default. The reference permits at most 12 dimension columns and warns that Kendall has higher complexity and can be slow on large datasets.

Those choices turn one apparently simple prompt into several independently testable decisions. “How does body mass correlate with flipper length, culmen length and culmen depth?” must identify one target and three comparison columns. A request for rank correlation must not silently fall back to the Pearson default. A request broken down by geography or product must map those fields to dimensions rather than additional metrics.

A minimum acceptance set

The desk’s proposed benchmark is small but stricter than grading only the final prose:

1. Hold the table fixed and vary only the named target, comparison columns and method.
2. Add dimension requests from zero through the documented maximum of 12, then test a thirteenth request for a clear refusal or bounded alternative.
3. Include both genuine NULL dimension values and rollup rows. Google’s output uses NULL for both cases in dimension columns; the separate segment field distinguishes them.
4. Grade the generated SQL, selected function arguments and returned metadata—not merely whether the narrative sounds plausible.

That last check matters because ML.CORRELATION returns segment_size and segment_proportion alongside the coefficient. Google’s own nine-row example reports an overall Pearson coefficient of 0.698, a five-row USA segment at 0.968 and a four-row null-country segment at 0.990. Those are illustrative documentation values, not evidence about agent accuracy.

What Google has—and has not—shipped

BigQuery agents can be configured with metadata, instructions and verified queries. Google says direct conversations omit that agent context and can therefore be less accurate. For recurring correlation questions, a verified query can lock down the intended target, dimensions and method; free-form chat still needs the function-selection test above.

The September release expands what the agent can execute. The next missing artifact is an evaluation set that proves the agent chooses the right statistical contract before it explains the number.