Benchmark leakage
Data contamination · Test-set leakage
When knowledge of the test set seeps into the system being tested, so a score measures memory of the benchmark instead of skill.
Background conceptGlossary entry
On this page
Benchmark leakage happens when something about the test set reaches the system under test: the questions, the expected answers, the document names, or just the shape of the test cases. The score then measures how well the system recognizes the benchmark, not how well it works.
It rarely looks like cheating. A typical case: extraction scores poorly on a scanned form in the test set. An engineer looks at the failure, notices the form has a field called “Inspection date”, and adds a rule to look for dates near that label. The score jumps. On the next customer’s forms, which say “Date of visit”, nothing changed. The fix was tuned to one input the team had already seen.
Common routes in
- Prompts, heuristics or schemas that contain fixture names, field names or expected values.
- Choosing between variants by their score on the held-out set, then reporting that same score.
- Language models trained on public benchmarks, which can recite answers they never retrieved.
- Duplicates: the same document, or a near copy, in both the tuning and test sets.
How DocSlurp guards against it
DocSlurp’s evaluation rules treat test inputs as held-out evidence, never as requirements. Held-out splits sync as separate datasets that tuning tools refuse to read (assertTuningReadable in core/scripts/eval-sync/langfuse.mjs). A separate integrity gate, core/scripts/evaluation-integrity-judge.mjs, requires the dataset to live outside the repository, checks label hashes and provenance, and scans product code for case identifiers, label names and distinctive expected values. A result must pass that gate before it can justify a change, and a leak discovered later invalidates the run.
The rule of thumb: if a change would make sense to someone who had never seen the test set, it is probably safe.
In DocSlurp
Background concept
DocSlurp keeps held-out splits unreadable by tuning tools (assertTuningReadable in core/scripts/eval-sync/langfuse.mjs) and screens product code for benchmark identifiers, label names and expected values with core/scripts/evaluation-integrity-judge.mjs, a gate separate from quality scoring.