Score normalization
Min-max normalization · Score calibration
Rescaling scores from different retrievers or queries onto a shared range so they can be compared, combined or thresholded.
In DocSlurpGlossary entry
On this page
Different retrievers produce scores on different scales. A BM25 score has no fixed maximum and grows with query length; a cosine similarity sits between −1 and 1 and often bunches in a narrow band. Before you can add them, compare them or apply one threshold to both, you have to put them on a common scale.
The most common method is min-max: (score − min) / (max − min) over one query’s results, which maps the best hit to 1 and the worst to 0. Take three results from each retriever for “When does the London office open?”:
| Rank | BM25 | Min-max | Cosine | Min-max |
|---|---|---|---|---|
| 1 | 14.2 | 1.00 | 0.83 | 1.00 |
| 2 | 9.1 | 0.54 | 0.80 | 0.25 |
| 3 | 3.0 | 0.00 | 0.79 | 0.00 |
The scales now match, but look at what happened: a cosine gap of 0.01 between ranks 2 and 3 became a gap of 0.25. Min-max stretches whatever spread the list happens to have. Z-scores (subtract the mean, divide by the standard deviation) have the same weakness on short lists.
The gotcha
Per-query normalization makes the best result of every query a 1.0, including a query where nothing relevant exists. It fixes scale, not meaning, so you can’t use a normalized score as a “this is relevant” threshold across queries. For that you need a score calibrated against what unrelated text looks like. Many systems skip score mixing entirely and fuse by rank with RRF.
DocSlurp does both. Hybrid search fuses by rank. The relevance floor converts vector scores back to cosine and rescales them above a noise baseline measured for the embedding pipeline, so 0 means “indistinguishable from unrelated text” on every query (core/server/src/search/relevance.ts). Search confidence, by contrast, divides by the top score (core/server/src/search/confidence.ts), so it is relative to the query, exactly the trap above.
In DocSlurp
In DocSlurp
Hybrid search sidesteps score mixing with rank fusion (core/server/src/search/rrf.ts). For the relevance floor, vector scores are converted back to cosine and rescaled above a measured noise baseline for that embedding pipeline (core/server/src/search/relevance.ts). Search confidence divides by the top score, so it is relative to the query (confidence.ts).