OpenAI’s data agent treats pipeline code—not schemas—as the semantic layer

OpenAI has published the architecture of the internal data agent it uses across a platform serving more than 3,500 employees, 600 petabytes of data and 70,000 datasets. The scale is striking, but the useful lesson for other data teams is smaller and more practical: schemas and query history are not enough to explain what a table means. The agent also reads the code that built it. (OpenAI)

Six context layers, not one giant prompt

The system builds a normalized context record in a daily offline pipeline. It combines schema and lineage metadata, historical query patterns, human-authored table descriptions, Codex-derived code analysis, institutional knowledge from systems such as Slack and Notion, and saved user corrections. At runtime it retrieves only relevant embedded context and can issue live warehouse queries when stored information is missing or stale. (OpenAI)

That separation matters for NL2SQL deployments. A column name can describe shape, while pipeline code carries inclusion rules, freshness assumptions, event semantics and transformation logic. OpenAI says code enrichment lets its agent distinguish superficially similar tables—for example, whether a dataset includes only first-party ChatGPT traffic—and refresh that understanding automatically. (OpenAI)

OpenAI’s retrieval design also avoids pushing all 70,000 datasets into every prompt. Its daily pipeline converts the enriched records into embeddings; query-time RAG selects the relevant subset, while live warehouse inspection handles unresolved questions. That is a concrete answer to a common large-schema failure mode: context selection becomes a maintained data product rather than an improvised prompt. (OpenAI)

Trust comes from execution and review

The evaluation loop uses curated natural-language questions paired with manually authored “golden” SQL. Generated SQL is executed, and the system compares both the query and returned data with the expected result before an Evals grader produces a score and explanation. OpenAI says these checks run continuously during development and act as production canaries, avoiding the false failures that exact SQL-string matching would create. (OpenAI)

Security is inherited rather than recreated in the agent. Access is pass-through: users can query only tables they already have permission to access. Answers summarize assumptions and execution steps and link to underlying results for inspection. This does not eliminate errors—OpenAI explicitly says the system can make mistakes—but it keeps authorization at the data layer and gives reviewers an evidence trail. (OpenAI)

The company also narrowed overlapping tools after finding that redundant choices confused the agent, and found that rigid step-by-step prompting degraded results. Its stated preference is high-level goal guidance, fewer ambiguous tools and model-selected execution paths. (OpenAI)

What operators should copy

The transferable pattern is not “deploy a chatbot over the warehouse.” It is to maintain six separate assets: access-controlled metadata, query examples, expert annotations, code-derived semantics, retrievable institutional context and editable memory. Pair those assets with executed-result evaluation and permission pass-through. OpenAI’s broader data-team account says its agent can save corrected definitions or filters for future questions, while humans still review evidence, challenge assumptions and test alternative explanations before acting. (OpenAI data-team account)

One boundary is important: this is a self-reported architecture, not an independent benchmark, and OpenAI does not publish an accuracy rate in the post. Teams should copy the observable controls—code-aware context, executable evals, scoped memory and inherited permissions—without treating internal adoption as proof that the same design will transfer unchanged to their warehouse.