A 0.8B text-to-SQL adapter reaches 90%—inside the synthetic-row test behind the number

A new LoRA adapter for Qwen3.5-0.8B reports 90% execution accuracy on its held-out text-to-SQL test, compared with 44% for the pinned base model. The jump is substantial, but the useful lesson is not that a 0.8B model has “solved” text-to-SQL. It is how narrowly—and unusually transparently—the result is defined.

What was actually trained

The publisher fine-tuned 3.608 million of 752.392 million parameters on an Apple M1 Mac with 8 GB of unified memory. Training used 2,000 examples from the sql-create-context dataset, with another 200 for validation and 100 for testing. The source dataset has no predefined split, so the project shuffled it once with a fixed seed and reports zero exact question-and-schema overlap among the three resulting subsets.

The adapter targets a specific contract: an English question plus CREATE TABLE statements goes in, and one SQLite SELECT query comes out. It is not a standalone checkpoint and must be loaded over the exact pinned MLX base revision. The card explicitly excludes other SQL dialects and production-readiness claims.

Why 90% does not mean 90% in production

The primary metric instantiates each test schema in three in-memory SQLite databases, each with 25 seeded random rows per table. A generated query passes only when its result multiset matches the reference query on all three databases. That is stronger than checking whether SQL merely parses: the tuned adapter’s valid-SQL rate was 99%, while execution accuracy was 90%.

But synthetic rows can still make two non-equivalent queries look equivalent by accident. The generated values are also biased toward literals in the reference query. The publisher acknowledges both that limitation and the uncertainty from a test set of only 100 examples, estimating a 95% interval of roughly plus or minus 10 percentage points.

The other metrics help interpret the result. Exact and normalized exact match were both 81% for the adapter, versus 3% and 18% for the base. That combination suggests much of the measured gain came from learning the dataset’s expected SQL form, not only from producing alternative equivalent queries.

The right use of this release

This is best treated as a reproducible laptop-scale fine-tuning recipe. The card pins the base revision, dataset revision, software versions, split procedure, prompt format, training settings and evaluation method; it also describes manifests and per-example evaluation records in the producing project.

For practitioners, the next test should replace the synthetic 100-example slice with representative schemas, real value distributions, ambiguous questions and the target SQL dialect. The adapter demonstrates that a small model can learn a constrained schema-to-SQL contract cheaply. It does not establish robustness on long schemas, nested queries, unseen domains or live enterprise data—and the release says so plainly.