VikingRAG’s biggest token saving starts with 1,000 synthetic history questions
The paper’s 5.1%–32.5% result is promising, but it measures a warmed system whose evidence checker still skips needed escalation on more than 5% of queries in many datasets.
A new structured-document retrieval paper reports the kind of efficiency gain data-agent builders want: VikingRAG uses 11.6%–51.9% of the query-time tokens consumed by the two highest-accuracy baselines, while its history-enhanced VikingRAG-E+ variant lowers that range to 5.1%–32.5%. The evaluation is substantial—but the smallest number is not a cold-start result.
What the test actually measures
The authors compare VikingRAG with eight RAG systems on six structured-document question-answering datasets. Collections reach 8.78 million tokens and include PDF, Markdown, DOCX and TXT files. Every system uses the same answer-generation model and prompt; accuracy is judged by an LLM against gold answers, followed by expert review of inconsistent or ambiguous decisions. Most experiments use DeepSeek-V4-Pro Preview, with selected VersionQA experiments repeated on GPT-5.5, Seed-2.0 and GLM-4.7.
The base design keeps document hierarchies outside the prompt and exposes selected directory segments through search, list, grep and read operations. That cold-start VikingRAG configuration produces the 11.6%–51.9% token ratio against the most accurate baselines.
VikingRAG-E+ adds two layers: it reuses prior retrieval traces as “experience edges,” then attempts a one-round retrieval and escalates to multi-round agentic search only when an evidence-sufficiency check says more retrieval is needed. In the benchmark, those experience edges are warmed with 1,000 independently generated, document-grounded historical questions per dataset by default. The evaluation questions and gold answers are withheld during that warm-up, but the history still comes from the same document collection.
Read the 5.1% carefully
The warm-up is a legitimate simulation of a recurring production workload, not test-set leakage as described by the authors. It is also an operating condition that should travel with the headline result. A new deployment without useful query history should expect the base system’s range, not automatically the 5.1%–32.5% range.
There is a second caveat. VikingRAG-E+ uses only 49.8%–81.1% of VikingRAG-E’s tokens and 73.1%–89.8% of its latency on average, but its evidence checker’s “false-NoEscalation” rate still exceeds 5% on many datasets. That means it sometimes stops after one round when the available evidence is insufficient. The paper says this does not cause a proportional accuracy drop because multi-round retrieval also fails on many of those hard questions.
That is an important distinction for enterprise data agents. A lower average token bill can come from better retrieval reuse, but also from deciding not to search again. Evaluation cards should therefore report cold and warmed token cost separately, disclose the history size and similarity threshold, and pair average accuracy with the rate of wrongly suppressed escalation. VikingRAG’s result is strongest when presented as a three-part benchmark—quality, cost and routing error—not as one universal compression ratio.
sources
comments · 0