Multi-hop retrieval
Iterative retrieval · Multi-hop question answering · Retrieval chain
Answers a question whose second search depends on a fact found by the first: retrieve, read out the bridging fact, then search again with it.
In DocSlurpIllustrated explainer
On this page
“Who approves the jet for the exec leading the London launch?”
Hop 1 · search
who approves the jet for the exec leading the London launch
launch-review.pdf p.5“The COO leads the London launch.”
Bridge · read
Who leads the launch? COOadded to the next query
Hop 2 · search
who approves jet charter requests for the COO
travel-desk-guide.pdf p.7“Charter requests for the COO are approved by the CFO.”
Both passages found. Answer: the CFO, citing launch-review.pdf p.5 and travel-desk-guide.pdf p.7.One passage found. The question never says “COO”, so the travel desk guide isn’t retrieved and the approver stays unknown.
The problem: the search you need can’t be written yet
“Who approves the jet for the exec leading the London launch?”
Two documents hold the answer. launch-review.pdf p.5 says “The COO leads the London launch.” travel-desk-guide.pdf p.7 says “Charter requests for the COO are approved by the CFO.” Neither mentions the other’s subject. The launch review never discusses approvals; the travel guide never mentions London.
A single search finds the first page, because the question shares “London launch” with it. It can’t find the second, because the word that links them, “COO”, isn’t in the question. No amount of candidate depth, rewriting or reranking fixes that reliably: the missing term is a fact, and it lives in the corpus.
The idea: retrieve, read, retrieve again
Multi-hop retrieval runs searches in sequence and lets each one use what the last one found:
- Hop 1. Search with the question. Top hit: the launch review.
- Bridge. Read the results and pull out the fact that connects to the next step: “COO”. This is called the bridge entity.
- Hop 2. Search again with the bridge added: “who approves jet charter requests for the COO”. Top hit: the travel desk guide.
- Answer from both passages, citing both.
The figure steps through it. Switch to a single search to see which passage never arrives.
The term comes from question-answering research. HotpotQA (2018) collected questions that need two Wikipedia paragraphs and named two kinds: bridge questions like this one, and comparison questions, which need two independent lookups and are better handled by query decomposition.
Ways to take the second hop
- Extract and re-query. A model names the bridge entities and they are added to the query. Cheap and easy to log.
- Reason between hops. IRCoT interleaves retrieval with chain-of-thought: the model writes one reasoning sentence, that sentence becomes the next query, and so on until it can answer.
- Let the model drive. Agentic RAG gives the model a search tool. Multi-hop falls out of the loop, without a fixed hop count.
- Follow a graph. If a knowledge graph was built at ingest, graph expansion walks
London launch → led by → COO → charter approved by → CFOwith no model call at query time. - Follow the document. Some hops are explicit references: “see Section 4.2”, “as amended by Schedule B”. Resolving the reference is a lookup, not a search.
Not every miss needs a second search
Before adding hops, check where the missing evidence actually sits. In a sealed replay of our own retrieval evals, we traced five passages that search had missed. Three were directly next to a chunk that search had returned, one was four chunks away, and one needed a cue about its position in the document. On a separate set of 102 questions over investor documents, 80 of 264 missed pages were within two pages of a page that was returned.
Those are small samples from two corpora. They still suggest that a good share of “second hop” evidence is a neighbor, and reading the surrounding page (small-to-big retrieval) is far cheaper than another model call and another search.
Where it breaks
- Errors compound. Every hop trusts the one before. If hop 1 returns a page about the Paris launch, the bridge is the wrong executive, and hop 2 faithfully retrieves that person’s approver. The final evidence looks coherent and answers a different question.
- Drift. Each added bridge term pulls the query away from the original need. Keep the original question in every hop’s query, and cap the number of bridge terms.
- Latency is serial. Hops can’t run in parallel. Each waits for a search and usually a model call, so three hops cost roughly three times the delay of one.
- No natural stopping point. Without a rule, a loop keeps hopping. Stop when a hop returns nothing new, when the next query repeats an earlier one, or at a fixed hop count, then answer or abstain.
- Version mix-ups. A hop can land in an older or amended copy of a document. A chain that starts in the 2026 agreement should not finish in the 2024 one. See document versioning.
When to use it
Use multi-hop when questions routinely connect documents through a named thing: a person and their role, a supplier and its contract, a part and its specification. Skip it when questions are single lookups, or when the parts are independent and can be searched in parallel.
In DocSlurp
iterativeMultiHop is off by default and can be set per project or per chat request. It runs up to three searches. After each one, a model reads the top six passages and returns up to six bridge entities that the evidence explicitly supports; they are appended to the original question to form the next query. The loop stops early when a hop returns nothing, no bridge is found or the next query repeats. The hop lists are fused with RRF, and the response includes a retrievalChain listing each hop’s query, bridge entities and result count, so you can see where a chain went wrong.
Chat’s default search agent covers the same ground a different way: it reads each result and writes the next query itself.
In DocSlurp
In DocSlurp
iterativeMultiHop (off by default) runs up to three searches. After each, a model names up to six bridge entities from the top six passages; they are appended to the original question for the next hop. The hop lists are fused with RRF and the chain is returned as retrievalChain (server/src/search/session-planner.ts).
Related ideas
Sources
- Yang et al. (2018): HotpotQA, a dataset for diverse, explainable multi-hop question answering
- Trivedi et al. (2023): Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions (IRCoT)
- Xiong et al. (2021): Answering complex open-domain questions with multi-hop dense retrieval