BigQuery can now summarize millions of rows—but AI.AGG makes prompt design part of the query plan

Google Cloud has put a generative aggregate inside BigQuery. The new AI.AGG preview accepts a natural-language instruction, batches rows to fit a Gemini context window, runs those batches in parallel and recursively feeds the intermediate summaries back through the same instruction until one string remains for each SQL GROUP BY group.[^1]

That is a useful abstraction, but it is not a deterministic SQL aggregate in the conventional sense. Teams adopting it should treat the prompt, null handling and reduction behavior as part of the data pipeline—not as presentation glue.

What the engine now handles

BigQuery performs row-size validation, context-safe batching, parallel Vertex AI inference and recursive synthesis inside the database. The function supports text as well as multimodal inputs through Object Tables and OBJ.GET_ACCESS_URL. Google positions log diagnosis and category discovery as representative workloads: summarize groups of raw events, or derive a JSON label set that can feed a row-level classifier.[^1]

This removes custom map-reduce plumbing, but it does not remove the model from the execution semantics. Every reduction stage uses a full-weight LLM, and the same instruction is applied repeatedly. The final answer therefore depends on how well that instruction survives compression across multiple levels.

Three production traps

First, token use can exceed the source data’s token count because intermediate outputs are sent through Gemini again. Google recommends filtering with WHERE and LIMIT, estimating with AI.COUNT_TOKENS, and considering Gemini Context Caching for large static inputs.[^1]

Second, missing data can disappear quietly. If any field inside a STRUCT is null, the entire row is treated as null and discarded; Google recommends wrapping nullable fields with IFNULL. Transient batch errors can also cause rejected rows to be skipped rather than failing the whole query.[^1] For audit-sensitive analysis, that means teams need explicit input and output counts around the aggregate.

Third, preview limits shape the query design. Individual rows cannot exceed 10 MiB; the final result is capped at 10,000 tokens per group; multimodal arrays should stay below 10 images per row. Google recommends no more than 20 million input rows and 1,000 output groups per query.[^1]

The practical takeaway

AI.AGG is best understood as a managed recursive summarization plan with SQL grouping—not a magical semantic SUM. Use strict output contracts, delimit fields, define failure values and preserve reconciliation metrics. The preview makes large-scale synthesis easier to run; it does not make the synthesis inherently complete or repeatable.

[^1]: Google Cloud, “Semantic aggregation over millions of rows in BigQuery with AI.AGG,” Nov. 5, 2026.