Git4Data wins BranchBench—until the workload stops being delta-sized

A new BranchBench result makes a useful distinction that headline speedups usually hide: database branching is cheap when an agent changes a bounded delta, but whole-table work can erase much of that advantage.

The Git4Data paper evaluates a MatrixOne prototype against DoltDB on four BranchBench workflows at scale factor 100, or approximately 47 million rows. Each workflow uses five concurrent agents over 20 steps. On warm runs, Git4Data completes data_cleaning in 58.6 seconds versus 1,084.2 seconds for DoltDB, an 18.5× speedup. The same comparison is 122.1 versus 1,925.6 seconds for software_dev (15.8×), 39.8 versus 410.2 seconds for mcts (10.3×), and 198.9 versus 1,677.3 seconds for failure_repro (8.4×).

The scaling test is the real result

The paper then increases the data from scale factor 100 to 1,000. software_dev rises from 127.5 to 366.3 seconds, a 2.9× slowdown, while data_cleaning rises from 99.3 to 322.5 seconds, or 3.2×. That is sublinear against a 10× increase in data size because these workflows write bounded per-step deltas.

failure_repro behaves differently. Its runtime grows from 199.0 to 2,685.7 seconds, a 13.5× slowdown, because its repair path scans and rewrites the full table. The result is a warning against treating “metadata-only branching” as a universal agent-workload speedup: it removes the cost of creating a branch, not the cost of SQL that touches everything inside it.

The 1,000-agent simulation reinforces the boundary. At scale factor 100, Git4Data finishes in 400 seconds while DoltDB does not finish within two hours. At scale factor 1,000, Git4Data takes 600 seconds. The paper attributes the remaining bottleneck to shared compute and I/O from concurrent branch-local executions rather than branch creation itself.

What the benchmark does—and does not—establish

The microbenchmark shows why the delta-sized cases favor Git4Data. Cloning a 100 GB lineitem table takes 0.20 seconds and 314 KB with a primary key, versus 114.6 seconds and 34 GB for a materialized insert. For one million random row updates, the built-in primary-key diff takes 3.27 seconds versus 431.50 seconds for the SQL equivalent; merge takes 16.13 versus 471.16 seconds.

But this is an author-run evaluation of a MatrixOne implementation against DoltDB, not an independent reproduction. The paper also names unresolved limits: diff and merge require compatible schemas; conflict resolution is row-level; and sustained snapshots can accumulate retained storage. The defensible takeaway is narrower than “Git4Data is 18.5× faster.” BranchBench shows that storage-native branches can make speculative agent workflows practical when changes remain sparse—and that benchmark reports should separate branch overhead from the work performed inside each branch.