Prompting database constraints is not enforcement: 120 SQL trials show why

The result

A new database-systems paper puts a concrete number on a familiar production risk: generated SQL does not reliably preserve invariants that the database itself cannot enforce. The authors tested Claude Sonnet 5 and GPT-5.6-Terra on six update scenarios, running each model five times per scenario under two prompt conditions. That produced 120 inspected generations: 60 without the constraints stated and 60 with all six constraints written into the prompt. Primary paper

Without the constraints in the prompt, all 60 generations applied no check at all. The outputs were bare INSERT or UPDATE statements that would violate the intended invariant. This is not execution accuracy on BIRD or Spider; it is a small, controlled writer-safety probe over a synthetic schema. Its value is the failure taxonomy, not broad model ranking. Primary paper

Explicit rules changed the SQL—not the guarantee

When the six constraints were supplied explicitly, 40 of 60 generations used a silent restriction such as WHERE NOT EXISTS or a count predicate. A violating write would affect zero rows rather than raise a constraint-specific error, leaving detection to the caller. Another 15 used check-then-raise patterns. Under PostgreSQL’s default READ COMMITTED isolation, those checks can race with concurrent transactions and admit write skew. Four generations still provided no guard, and one returned an empty response. Primary paper

Only one of the 60 explicitly constrained trials used PostgreSQL SERIALIZABLE isolation with retries, the paper’s general solution for the concurrency race. Other outputs tried table locks, proxy-row locks or advisory locks. The strongest table-lock answer serialized writers across as many as seven of eight tables for one insert; the weaker conventions only worked if every writer followed the same protocol. Primary paper

The prompt also expanded the generated program substantially. For the shared person_id constraint, average output length rose from 16 words without constraints to 153 words with them. More generated enforcement logic created more places for silent behavior, placeholders and inconsistent locking choices. Primary paper

What practitioners should take from it

The evidence does not establish that these two model families fail on every integrity task: there are only six scenarios, five samples per model and no reported execution-based pass rate. It does establish a narrower point cleanly. A natural-language constraint list is not equivalent to a transactional database constraint, especially when multiple writers can race. Primary paper

The paper’s prototype response is a PostgreSQL constraint-index extension for multi-relation invariants. Across its evaluated workloads, it beats the better correctness-preserving baseline by up to 2.41× at 256 concurrent clients. That systems result is separate from the model probe, but the connection is practical: keep generated SQL simple, and place durable invariants at an enforcement boundary shared by every writer. Primary paper