BigQuery’s own example shows why accuracy alone is a weak evaluation

BigQuery’s new ML.METRICS function makes model evaluation easier to run—and its own documentation offers a compact warning against reporting only the friendliest number.

Google’s September 10 release note introduced the Preview function for calculating classification or regression metrics from any table or query containing actual and predicted values, without first creating or referencing a stored BigQuery ML model. The function reference shows a 100-row news-classification query whose output is 0.84 accuracy, 0.33 precision, 0.28 recall, and 0.30 F1.

One result row, four different stories

A headline built around 0.84 would sound strong. The accompanying 0.30 F1 score says the same predictions perform far less evenly across labels. BigQuery documents that STRING labels are treated as multiclass classification—even if the data contains only two distinct labels—and that precision, recall and F1 are macro-averaged across classes. Each class therefore receives equal weight rather than being washed out by a large majority class.

That behavior is useful for evaluating analytics classifiers whose rare categories may matter disproportionately. It also means teams need to record the label type and averaging rule alongside the number. A BOOL task is scored differently: BigQuery treats it as binary classification and computes metrics for the positive (TRUE) class.

The silent denominator change

There is another benchmark trap in the specification: rows containing a NULL actual or predicted value are omitted from metric calculations. If every row is filtered out—or the input is empty—the function still returns one row, but all metric fields are NULL.

For repeatable evaluation, an ML.METRICS report should therefore include at least the input row count, the number of rows omitted for nulls, label data type, task type, and all returned metrics. None of that requires a new evaluation framework; it can be captured in the same SQL workflow that produces the metric row.

The function’s value is not that it chooses the right headline metric. It is that BigQuery now exposes the disagreement between metrics directly in SQL. The documentation’s 0.84-versus-0.30 example is a ready-made test of reporting discipline: if a dashboard shows only accuracy, the evaluation is incomplete even when the query itself is correct.