Production agent quality now needs three loops—not one bigger evaluation set

Enterprise teams are converging on three different quality loops for production agents. They are not substitutes: one cleans what the agent is allowed to believe, one discovers what repeatedly goes wrong, and one tests whether a proposed repair actually holds.

Salesforce’s first-year Employee Agent report describes the corpus loop. Its library of more than 1,000 internal articles contained overlapping policies and outdated text, which produced confusing or incorrect answers. Salesforce says it archived hundreds of pages, rewrote policy language and began using AI scoring plus case-data analysis to flag weak or missing knowledge. The company reports a 97.7% self-service rate after a year, but that operating metric is not an independent correctness evaluation. (Salesforce)

Conversion describes the trace loop. Its Periscope system does not treat a whole conversation as one success or failure. It anonymizes and compacts traces, splits them into single-intent “episodes,” assigns resolved, partial, failed or unclear outcomes, and only promotes a failure mode when the same mechanism recurs across independent episodes. That corpus view surfaced agents repeatedly recovering from missing sandbox packages and inefficiently looping over single-item tools for aggregate questions—problems that users rarely reported because the task still completed. (Conversion)

Block’s Moneybot supplies the replay loop. Its repair system turns a reported production issue into a synthetic staging scenario, runs a ten-sample baseline, expands to twenty when a failure appears only once, and parks tickets that do not reproduce. Candidate fixes are rerun with the same sample size before the system opens separate pull requests for the patch and its permanent regression case. Block reports more than 40 verified fixes over one quarter, while keeping merge and ticket-closing authority with people; those are company-reported results from its own environment. (Block)

Why NL2SQL teams need all three

A data agent can pass a replay test while still retrieving a deprecated metric definition. It can have a clean semantic corpus while quietly issuing slow one-table-at-a-time queries. And a trace cluster can identify a recurring wrong-join pattern without proving that a prompt or tool change reduced its failure rate.

The operating sequence is therefore:

1. Curate the truth: remove superseded definitions and make grain, joins, exclusions and ownership explicit.
2. Map production behavior: segment multi-intent conversations, preserve “unclear” outcomes and require recurring evidence before naming a failure mode.
3. Replay before shipping: reproduce the failure repeatedly, compare equal-size before-and-after samples, and attach the reproducer to the change.

This pattern changes what “evaluation” means. A static benchmark remains useful for comparing snapshots, but production quality is a maintenance system: the corpus supplies current truth, trace analysis sets priorities, and stochastic replay decides whether a repair is credible. Teams that collapse those jobs into one accuracy score will miss stale context, hidden recovery costs, or fixes that only got lucky once.