Recall@k
Recall at k · R@k
The fraction of all relevant passages for a query that appear in the top k results. It measures what retrieval found, not how it ordered it.
Background conceptIllustrated explainer
On this page
The problem: the answer was never in the room
A user asks “How fast can we get the Learjet to London?” and the chat answer is vague. You read the prompt and the model looks fine. Then you look at what retrieval passed it: ten passages, and the one that says “wheels-up within three hours of the assistant’s request” isn’t among them. The model never had a chance.
This is the most common RAG failure, and it is invisible if you only grade final answers. You need a number for the retrieval step on its own. Recall@k is that number.
The definition
For one query, with a set of passages known to be relevant (from relevance judgments):
recall@k = (relevant passages in the top k) / (all relevant passages)
Average it over the queries in your golden dataset to get the system’s recall@k.
Work through the figure. Four passages in the corpus are relevant to the question. Retrieval returns ten results, with relevant ones at ranks 1, 4 and 7. The fourth relevant passage, travel-desk-guide.pdf p. 7, is nowhere in the list.
| k | Relevant in top k | Recall@k | Precision@k |
|---|---|---|---|
| 1 | 1 | 1/4 = 0.25 | 1/1 = 1.00 |
| 3 | 1 | 0.25 | 1/3 = 0.33 |
| 4 | 2 | 2/4 = 0.50 | 2/4 = 0.50 |
| 5 | 2 | 0.50 | 2/5 = 0.40 |
| 7 | 3 | 3/4 = 0.75 | 3/7 = 0.43 |
| 10 | 3 | 0.75 | 3/10 = 0.30 |
Three things show up, and the slider makes them visible:
- Recall only moves when a relevant result crosses the line. It is a staircase. Ranks 2, 3, 5, 6, 8, 9 and 10 add nothing.
- Recall stalls at 0.75. No k up to 10 reaches 1, because one relevant passage was never retrieved. Recall measures retrieval’s coverage, and no amount of reordering changes coverage.
- Precision zig-zags down. Precision@k divides by k instead, so each irrelevant result drags it down and each relevant one bumps it up.
Why RAG cares about recall first
In a search page, a user scrolls. In RAG, the top k is the model’s world: only those passages enter the prompt. A relevant passage at rank k + 1 might as well not exist. That makes recall@k the ceiling on answer quality. A reranker can reorder what retrieval found, and context assembly can trim it, but neither can add the travel desk page back.
This suggests a two-stage habit:
- Measure recall at the candidate depth (say k = 50 or 100) to see what retrieval can reach at all. If it is low there, fix retrieval: chunking, hybrid search, query rewriting.
- Measure recall at the context depth (say k = 5 or 10) to see what the model actually gets. If deep recall is high but shallow recall is low, the passages exist but rank poorly: fix ranking.
Where it breaks
- It needs the full set of relevant passages. The denominator assumes you know everything relevant. You never do. Unlabeled relevant passages make recall look higher than it is (the denominator is too small) or make a new retriever look worse (it finds passages nobody labeled).
- It ignores order inside k. A relevant result at rank 1 and at rank 10 count the same. When position matters, use nDCG or mean reciprocal rank.
- Averages hide all-or-nothing questions. A question that needs two premises, with one found, scores 0.5, but the model may still write a wrong answer. A per-question pass rule (“every premise in the top k”) is stricter and closer to whether the answer is possible.
- Passage identity is fragile. If relevance is labeled by chunk ID, re-chunking the corpus breaks the labels. Label by quoted text and page instead, and match on that.
- It says nothing about the answer. High recall with a bad prompt still produces bad answers. Pair it with faithfulness and answer checks.
When to use it
Use recall@k as the first metric for any retrieval change: new embeddings, chunk sizes, fusion settings, filters. Report it at two depths (candidate and context), next to a rank-aware metric. Read the failures, not just the average: a missing passage is usually a chunking, extraction or vocabulary problem you can see by eye.
DocSlurp’s retrieval scorer applies the strict per-question version: a case is correct only when every gold premise appears in the top k (default 10) with matching quoted text on a cited page (core/scripts/retrieval-quality/eval.mjs). Search tuning gates on recall@5 and recall@10 alongside nDCG@10 and MRR (core/docs/evals/search-latency-tuning-2026-10-23.md).
In DocSlurp
Background concept
DocSlurp’s retrieval scorer is an all-or-nothing recall@k: a case passes when every gold premise appears with matching text on a cited page within k, default 10 (core/scripts/retrieval-quality/eval.mjs). Search tuning also gates on recall@5 and recall@10 (core/docs/evals/search-latency-tuning-2026-10-23.md).