BigQuery can now store an agent’s entire execution trace—but observability is not evaluation

Google Cloud has turned agent observability into a data-engineering problem. Its open-source BigQuery agent analytics solution captures requests, responses, tool calls, workflow events and errors, then streams them into BigQuery through the Storage Write API. The practical shift is important: an agent run is no longer only an application log or a vendor dashboard. It becomes a queryable event dataset that teams can join to operational and business records.

Capture and analysis are separate layers

Google provides two ways to instrument supported frameworks. Agent Development Kit applications can attach the BigQuery agent analytics plugin; LangGraph and LangChain applications can use a callback handler, currently marked preview. Custom frameworks can send structured events directly through the Storage Write API. BigQuery data agents can also enable built-in conversation logging.

Those integrations write the events. A separate open-source Python SDK reads them, reconstructs multi-turn sessions and evaluates traces. Google says the streaming path buffers and serializes rows asynchronously so observability work does not block the user-facing agent turn. That is the right architecture, but operators should still measure queueing, dropped events and ingestion cost under their own traffic rather than treating “non-blocking” as “free.”

A trace is more useful than a transcript

A transcript shows what the user and model said. The SDK is designed to rebuild causal chains that include nested model and tool calls. That distinction matters for data agents: a correct final answer can hide an unnecessary query, a failed retry or a tool invoked in the wrong order.

Google exposes both deterministic checks—latency, turn count and error rate—and semantic checks such as correctness and hallucination scoring. It also supports matching actual execution paths against expected “golden” trajectories and monitoring request distributions for regression or drift. Teams can query token use and latency directly with SQL, build dashboards, or apply BigQuery functions such as AI.SCORE and AI.GENERATE to logged interactions.

Logging creates a second governance surface

The same completeness that makes the dataset useful makes it sensitive. Raw requests, responses, tool arguments, errors and multimodal payloads can contain user data, retrieved records or intermediate reasoning artifacts that were never intended for broad analytics access. Google points to granular access controls, masking and encryption, but those controls do not choose retention periods, redact payloads or define who may inspect a failed session.

Operators should therefore treat the events table as a production data product: document its schema, classify fields, set retention and deletion rules, restrict raw-payload access, and expose derived metrics separately from full traces. Version the evaluator and its rubric alongside every score. Otherwise a rising “pass rate” may reflect a changed judge rather than a better agent.

The useful idea is not that BigQuery automatically proves an agent is good. It is that agent behavior can now be audited with the same tools used for any other event stream. Telemetry supplies evidence; the organization still owns the evaluation contract.