Skip to content

LearnRank & assemble

Reranking

Re-ranking · Second-stage ranking

A second pass that re-scores a short list of retrieved candidates with a slower, more careful model, then keeps only the best few.

In DocSlurpIllustrated explainer

On this page
Reranking narrows 200 candidates to the 8 a model readsA funnel: an index of 1,000,000 chunks (illustrative), first-stage retrieval returns 200 candidates, a reranker scores each query and passage pair, and the top 8 go to the model. Below, six candidates for "How fast can my assistant get the Learjet to London?". First-stage order: charter-agreement.pdf page 9, board-update-2026.pdf, expansion-forecast.xlsx, charter-agreement.pdf page 2, travel-desk-guide.pdf, deal-room-guide.pdf. Reranker scores: travel-desk-guide 0.94, charter-agreement page 2 0.71, board-update 0.38, charter-agreement page 9 0.22, expansion-forecast 0.15, deal-room-guide 0.05. The answer, travel-desk-guide.pdf page 3, moves from 5th to 1st. With a cut at the top 3, the answer was excluded before reranking and included after.Index1,000,000 chunksFirst-stage retrieval200 candidateskeyword + vector, millisecondsRerankerreads query + passage200 pairs scoredTop 8to the modelIndex size is illustrative. 200 in and 8 out are DocSlurp’s defaults when reranking is on.First-stage orderrerankerAfter rerankingscore1Rcharter-agreement.pdf p.9rates2Bboard-update-2026.pdf p.5results3Eexpansion-forecast.xlsxmodel4Acharter-agreement.pdf p.2readiness5Ttravel-desk-guide.pdf p.3booking6Ddeal-room-guide.pdf p.7M&A1Ttravel-desk-guide.pdf p.30.942Acharter-agreement.pdf p.20.713Bboard-update-2026.pdf p.50.384Rcharter-agreement.pdf p.90.225Eexpansion-forecast.xlsx0.156Ddeal-room-guide.pdf p.70.05keep the top 3
Retrieval only has to get the answer onto the list; reranking gets it to the top. Asked “How fast can my assistant get the Learjet to London?”, first-stage retrieval ranks the travel desk page that answers it 5th, below a rates clause that happens to repeat “Learjet”. A reranker that reads each question and passage pair moves it to 1st. Cut at three, the answer goes from missing to first. Scores are illustrative.

The problem: the answer is on the list, just not near the top

You ask “How fast can my assistant get the Learjet to London?” Search returns 200 candidates in a few milliseconds, and the passage that answers it is among them: travel-desk-guide.pdf, page 3, “Once the assistant books it, the Learjet is wheels-up from Teterboro within three hours and lands at Farnborough about 7h 10m later.” It sits at rank 5.

Above it are a rates clause from charter-agreement.pdf that repeats “Learjet”, the board’s record-year update and an expansion spreadsheet. A model that reads the top three never sees the answer and tells you the documents don’t say.

First-stage retrieval ranks this way because it has to be fast across every chunk you own. It compares things computed ahead of time: posting lists in an inverted index, or one stored vector per passage. The travel desk guide was turned into a single vector at ingest, long before anyone asked about London. That is enough to land it in the top 200, but not enough to place it first.

Two stages: cast wide, then read carefully

Reranking splits the job in two:

  1. Retrieve for recall. Use cheap methods (BM25, vector search, or both fused with RRF) to pull a few hundred candidates out of millions. The only goal is that the answer is somewhere in the pile.
  2. Rerank for precision. Run a slower scorer on each (question, passage) pair in that pile, sort by its score, and keep the top few.

The cost arithmetic is why this works. A scorer that reads the question and passage together can’t run over a million passages per query. Over 200 candidates it runs 200 times, and the shortlist is what the model reads.

Here is the zoom from the figure, with illustrative reranker scores:

Passage First-stage rank Reranker score New rank
T travel-desk-guide.pdf p.3 5 0.94 1
A charter-agreement.pdf p.2 4 0.71 2
B board-update-2026.pdf p.5 2 0.38 3
R charter-agreement.pdf p.9 1 0.22 4
E expansion-forecast.xlsx 3 0.15 5
D deal-room-guide.pdf p.7 6 0.05 6

Before reranking, the top three were R, B and E, and none of them answers the question. After, the top three are T, A and B, and the answer is first.

What can do the reranking

  • A cross-encoder. The common choice. One model reads the question and passage together and outputs a relevance score, so it can see that “how fast can … get” is answered by “wheels-up within three hours”.
  • A language model. You can prompt a general model to score each passage or sort a short list. It is flexible and much more expensive per candidate.
  • A lexical rescorer. Recompute BM25 over just the shortlist. It costs almost nothing and needs no model, but it can only reward shared words. It can’t tell that “charter aircraft” means “private jet”.
  • A trained combination of signals. Learning to rank fits weights over many features, such as BM25, vector similarity and freshness, from judged examples.

How DocSlurp does it

Reranking is on by default for project search. With reranking on and no explicit candidate depth, DocSlurp ranks up to 200 fused candidates and returns the project’s result count, 8 by default (core/server/src/search/retrieval-controls.ts, session-execution.ts).

The default reranker is lexical. rerankBM25 in core/server/src/search/rerank.ts scores each candidate with BM25 (k1 = 1.5, b = 0.75), computing term rarity from the candidate pool only, not the whole corpus. It adds a bonus when two query words appear next to each other in the passage, so “private jet” as a phrase beats the two words scattered. The field it writes is named crossScore, but no cross-encoder is involved.

A project owner can opt into a hosted learned reranker, Qwen/Qwen3-Reranker-0.6B on DeepInfra (core/server/src/search/hosted-rerank.ts). It sends candidates in batches of 25 and checks the project privacy mode and spend allowance before every batch. It records token usage per batch. If anything fails, such as a timeout, a missing key, a privacy block, or a query and passage longer than 30,000 bytes together, the search falls back to BM25 and records why in the session diagnostics. A request with rerank: false makes no reranker call. See retrieval controls.

Where it breaks

  • It can’t rescue what retrieval missed. If the answer is at rank 240 and you rerank 200, it stays lost. Deeper candidate lists raise the odds and the cost together.
  • Cost and latency grow with depth. A model reranker does work for every candidate on every query. Doubling the depth doubles that work.
  • Long passages get cut. Rerankers have input limits. A passage that is truncated before its key sentence gets scored on the part the model saw.
  • Training data matters. A reranker trained on web questions may misjudge spreadsheets, logs or contract clauses. Check it on your own queries with recall@k or nDCG before trusting it.
  • Its scores only order this query’s results. A 0.94 here and a 0.94 on another query don’t mean the same confidence. Don’t threshold across queries without calibration.

When to use it

Use reranking whenever the model or the user sees only a few results and a wrong top three costs you an answer. That covers most RAG. Keep the shortlist deep enough to catch the answer, for example 50 to 200, and tune depth against latency. Skip it when users scan long result lists themselves and every millisecond counts.

In DocSlurp

In DocSlurp

On by default: up to 200 fused candidates are re-scored and the top 8 kept (core/server/src/search/session-execution.ts, retrieval-controls.ts). The default reranker is local BM25 over the candidate pool (core/server/src/search/rerank.ts), not a neural model; a project can opt into a hosted learned reranker (hosted-rerank.ts). See retrieval controls.

Sources

Jump to