One data-agent log is not enough: keep evidence, trajectory and action ledgers separate

Enterprise data agents are becoming observable, but “log everything” is not an architecture. Three first-party updates this week point to a more useful design: keep evidence, trajectory quality, and operations/actions as separate ledgers that can be joined when an incident occurs.

1. Preserve the evidence attached to each inference

Google Cloud’s September 10 Data Cloud digest says Pub/Sub Single Message Transforms can now run inference against models hosted on the Gemini Enterprise Agent Platform. The model prediction is appended to the incoming event before that event moves downstream to systems such as BigQuery or Bigtable.

That is not a full provenance system by itself, but it demonstrates the right primitive: keep the inference result beside the event that produced it. For an analytics agent, the equivalent record should include the source event or query context, the model or rule version, and the produced classification or enrichment. Without that pairing, teams can see a wrong downstream result but cannot reliably reconstruct what the system knew at the time.

2. Diagnose the trajectory turn by turn

AWS’s Agent Evaluation Metric addresses a different layer. It scores multi-turn conversations per turn and separates the turn that caused an error from later turns that merely inherited it. The initial implementation decomposes correctness into truthfulness and completeness, applying the same structure to natural-language responses and tool calls. AWS explicitly describes the default 0.5 semantic-similarity threshold as a starting point rather than a tuned universal value.

The practical lesson is that the evaluation ledger should not collapse an interaction into one final pass/fail result. Store the selected tool, required and supplied parameters, per-turn verdict, failure reason and whether the failure is causal or inherited. That turns a bad answer into a debuggable sequence.

3. Separate runtime telemetry from the action audit trail

Snowflake’s internal InvoiceIQ makes the final distinction unusually explicit. Every pipeline run records per-stage latency, token usage for each Cortex AI call, and error and retry counts. Separately, every AI-driven or human action writes an event to an audit table. Snowflake says the first record explains runtime behavior and cost; the second records what happened in the business workflow.

Combining those into one undifferentiated log would make both questions harder. Engineers need to ask which stage became slow or expensive. Finance and compliance teams need to ask who or what approved, rejected, changed or submitted an invoice. The retention rules and access controls may also differ.

A minimum production contract

A production data agent should therefore emit three joinable records:

• Evidence: the governed input, retrieved context or event and the inference attached to it.
• Trajectory: each response or tool-call turn, its quality verdict and the first causal failure.
• Operations and actions: latency, tokens, retries and cost telemetry, plus a distinct immutable history of business actions.

The shared key matters as much as the separation. A request, conversation, workflow and source-event identifier should let an investigator move from a disputed answer to the responsible turn, then to the exact evidence and resulting action. Observability is not one dashboard; it is a chain of records that can survive a model change, a retry and an audit.