Skip to content

LearnEvaluate

Hit rate

Hit@k · Top-k accuracy · Success@k

The fraction of queries for which at least one relevant result appears in the top k. Each query scores 1 or 0.

Background conceptGlossary entry

On this page

Hit rate (also hit@k or top-k accuracy) counts a query as a success if at least one relevant result appears in the top k, then averages over queries.

Three test questions, k = 3:

Query First relevant rank Hit@3
“Are we on track to beat our FY2027 earnings?” 1 1
“How far above our margin target are we?” 3 1
“Who signs off on the jet?” none in top 10 0

Hit rate@3 = 2 / 3 = 0.67. It’s easy to explain (“two thirds of questions found something useful”) and matches how many RAG systems behave, since one good passage is often enough to answer.

The gotcha

Hit rate can’t see partial success. “Compare our FY2027 projection with this year’s record results” needs two passages: growth-plan-2027.pdf p. 12 and board-update-2026.pdf. A system that finds only the first scores a full hit, and the model writes half an answer. For multi-part questions use recall@k, or a stricter check like DocSlurp’s retrieval scorer, which passes a case only when every premise is found within k (core/scripts/retrieval-quality/eval.mjs).

Hit rate also ignores position within k. Rank 1 and rank 3 both count as 1; mean reciprocal rank tells them apart.

In DocSlurp

Background concept

DocSlurp’s retrieval scorer is stricter than hit rate: a case is correct only when every premise appears, with matching text on a cited page, within k (default 10) (core/scripts/retrieval-quality/eval.mjs).

Sources

Jump to