Query planning
Query planner · Retrieval planning · Retrieval strategy selection
Deciding, before or during retrieval, which searches to run for a question, in what order and at what depth, instead of sending every question through one fixed search.
In DocSlurpIllustrated explainer
On this page
“What is the cost code for N750MC?”
- Hybrid searchkeyword + vector, top 8
- Searches
- 1
- Model calls
- 0
- Latency
- one round trip
The answer is one passage and the question already contains its words. Anything more is wasted work.
“What do the Q3 slides say about London?”
- Extract filtertype = slides
- Hybrid search“Q3 London”, inside the filter
- Searches
- 1
- Model calls
- 0
- Latency
- one round trip
“Slides” is a constraint. As a filter it removes every non-slide passage before ranking.
“Compare the margin target in the growth plan with the margin in the Q3 memo.”
- Splittwo sub-queries
- Search part 1growth plan margin target
- Search part 2Q3 memo margin
- Reserve a slot per partthen fill by score
- Searches
- 2
- Model calls
- 0
- Latency
- searches in parallel
Two independent lookups. They run at the same time, and each keeps its own evidence.
“Who approves the jet for the exec leading the London launch?”
- Search hop 1London launch lead
- Read the bridge“the COO”
- Search hop 2jet approval for the COO
- Searches
- 2
- Model calls
- 1
- Latency
- steps wait on each other
The second search needs a fact from the first, so the steps must run in order.
“What is driving our growth heading into FY2027?”
- Summaries built at ingestper topic or section
- Ask each summarypartial answers
- Combineone cited answer
- Searches
- 0
- Model calls
- 2
- Latency
- searches in parallel
No passage holds the answer. The work was done at ingest; query time reads summaries.
search model call rule done at ingest
The problem: one search shape for every question
A retrieval pipeline is usually built around one question shape: a single fact that sits in one passage. Embed the question, run hybrid search, keep the top 8. That handles “What is the cost code for N750MC?” well.
Real questions come in other shapes, and the same pipeline fails each of them differently:
- “Compare the margin target in the growth plan with the margin in the Q3 earnings memo” needs two passages from two documents. One search tends to fill the top results with whichever half has more matching text.
- “Who approves the jet for the exec leading the London launch?” needs a fact you don’t have yet. No passage mentions both the launch and the approver.
- “What do the Q3 slides say about London?” contains a constraint (“slides”) that should remove passages, not nudge a ranking.
- “What is driving our growth heading into FY2027?” has no passage-sized answer at all.
Raising top-k helps none of these reliably. The fix is to choose a different set of searches for each shape. That choice is the query plan.
The idea: pick the searches before running them
The term comes from databases. A SQL planner reads a query, estimates how many rows each step would touch, and picks an order of index scans and joins. A retrieval planner does the same job with different operators:
| Question shape | Plan | Extra cost |
|---|---|---|
| One fact | One hybrid search | None |
| Explicit constraint | Extract a filter (self-query), then search inside it | A rule or one model call |
| Several independent parts | Query decomposition: one search per part, in parallel | One search per part |
| Vague or oddly worded | Query rewriting, HyDE or multi-query retrieval | One model call, more searches |
| Specific case of a general rule | Step-back prompting: search the specific and the general | One model call, one search |
| A chain of facts | Multi-hop retrieval: search, read, search again | Serial searches and model calls |
| The whole corpus | Summaries prepared at ingest (GraphRAG, RAPTOR) | Paid at ingest |
The figure shows the plan for each shape. Pick a question and compare how many searches and model calls it spends.
Who does the planning
There are three places to put the decision.
- Rules. Patterns in the question trigger a plan: a quoted file name becomes a filter, “ vs “ triggers a split. Rules are fast, free and predictable, and they miss anything phrased differently.
- A classifier. A small model sorts the question into a shape first. Adaptive-RAG trains one to choose between no retrieval, one retrieval and an iterative loop.
- The answering model itself. In agentic RAG the model gets a search tool and plans as it goes. This is the most flexible option and the most expensive, because every step is a model call that rereads everything found so far.
Parallel or serial
Two plans with the same number of searches can feel very different. Independent searches (the parts of a compound question, a rewrite alongside the original) run in parallel, so latency stays close to one search. Dependent searches (hop two needs hop one’s result) must run in order, and each hop usually waits on a model. A plan is worth drawing as a small graph: what can start now, and what is waiting on something.
The baseline every plan must beat
A plan spends extra searches, and sometimes extra model calls. The same budget could have gone to plain depth: a larger candidate depth and a reranker. So the fair comparison for any plan is the default search at equal work, not the default search as it ships.
We learned to be strict about this. In a DocSlurp diagnostic of 384 known-passage probes, a single keyword query put the known passage in the top ten 206 times. Fusing three result lists did so 183 times, with 4 gains and 27 losses. The two arms did unequal backend work, so this isn’t a ranking of the methods, but it does show that more searches can return less.
Where it breaks
- Misclassification. A planner that splits a question which wasn’t compound, or invents a filter nobody asked for, does worse than no planner. A wrong filter excludes the right passage outright.
- Cost without a ceiling. Plans that loop need a step budget and a stopping rule, or they keep searching for evidence that isn’t there instead of abstaining.
- Merging undoes the plan. Running the right sub-searches and then fusing everything into one ranked list can bury the passage that only one sub-search found.
- Invisible plans. When an answer is wrong, the first question is which searches ran. Log the plan and the effective queries with the result.
In DocSlurp
Search has a rule-based planner. selfQuery extracts filters before retrieval. Then at most one multi-search mode applies, checked in this order: queryDecompose, stepback, iterativeMultiHop. All are off by default, so a plain search is one hybrid retrieval. Chat is different: its default search agent sees the conversation and plans its own queries within a step budget.
In DocSlurp
In DocSlurp
A rule-based planner applies at most one multi-search mode per request: queryDecompose, then stepback, then iterativeMultiHop, all off by default (server/src/search/session-planner.ts). selfQuery extracts filters first. In chat, the default search agent plans its own queries.