Query decomposition
Sub-question decomposition · Compound question splitting
Splits a question that asks for several independent things into sub-queries, searches each one, and makes sure every part keeps its own evidence in the final context.
In DocSlurpIllustrated explainer
On this page
“Compare the margin target in the growth plan with the margin in the Q3 earnings memo.”
Part 1 margin target in the growth plan
- Agrowth-plan-2027.pdf p.13Operating margin target: 20%
- Xboard-update-2026.pdf p.7Margins improved across the year
- Yexpansion-forecast.xlsx MarginsMargin by market, FY2027
Part 2 margin in the Q3 earnings memo
- Bq3-earnings-memo.pdf p.3Q3 operating margin reached 24%
- Xboard-update-2026.pdf p.7Margins improved across the year
- Yexpansion-forecast.xlsx MarginsMargin by market, FY2027
Fused with RRF (k = 60) 3 slots
- Xboard-update-2026.pdf p.7generic · fused 0.0323
- Yexpansion-forecast.xlsx Marginsgeneric · fused 0.0317
- Agrowth-plan-2027.pdf p.13evidence for part 1
Parts with evidence: 1 of 2
One slot reserved per part 3 slots
- Agrowth-plan-2027.pdf p.13evidence for part 1
- Bq3-earnings-memo.pdf p.3evidence for part 2
- Xboard-update-2026.pdf p.7generic · fused 0.0323
Parts with evidence: 2 of 2
The problem: one question, two answers, one ranking
“Compare the margin target in the growth plan with the margin in the Q3 earnings memo.” A person reads that as two lookups. A retriever reads it as one string.
The embedding of the whole question points somewhere between “growth plan margin target” and “Q3 memo margin”. The passages nearest that midpoint are the ones that talk about margins in general: a board update, a forecast sheet. Keyword search has the same lean, because a passage that mentions “margin”, “plan” and “Q3” in passing matches more query words than the one page that states the target. The two passages you need are each a strong match for half the question and a mediocre match for the whole.
The idea: split, search each part, keep evidence for each
Query decomposition breaks the question into sub-queries that can each be answered alone:
- Split. “margin target in the growth plan” and “margin in the Q3 earnings memo”.
- Search each part. The parts are independent, so the searches run in parallel and latency stays close to one search.
- Merge. Build one context that holds evidence for every part.
Splitting can be done by rules (conjunctions, “vs”, “compare”, sentence breaks) or by a model asked to list the sub-questions, as in self-ask and decomposed prompting. Rules are free and miss natural phrasing. Models catch more and sometimes split a question that was never compound.
Decomposition is for parts that don’t depend on each other. If the second part needs the answer to the first (“who approves the jet for the exec leading the launch?”), that is multi-hop retrieval. If you are rephrasing one need several ways, that is multi-query retrieval.
Step 3 is where it goes wrong
The usual merge is reciprocal rank fusion. RRF rewards passages that appear in several lists. That is the right bet when the lists are different views of the same need. Here the lists answer different needs, so the passage each part depends on appears in exactly one list.
Work it through with k = 60 and the lists in the figure:
| Passage | Part 1 rank | Part 2 rank | Fused score |
|---|---|---|---|
X board-update-2026.pdf p.7, margins in general |
2 | 2 | 1/62 + 1/62 = 0.03226 |
Y expansion-forecast.xlsx Margins sheet |
3 | 3 | 1/63 + 1/63 = 0.03175 |
A growth-plan-2027.pdf p.13, the 20% target |
1 | — | 1/61 = 0.01639 |
B q3-earnings-memo.pdf p.3, the 24% margin |
— | 1 | 1/61 = 0.01639 |
Both sub-searches did their job: each ranked its answer first. After fusion, the two generic passages lead, and A and B tie for third. With three slots, one of the two answers is cut, and which one is arbitrary. With two slots, both are.
The alternative is to reserve room per part: take the best result of each sub-query first, then fill what is left by fused score. The same lists give A, B, X. The research name for this is aspect-aware diversification (xQuAD treats each sub-query as an aspect the result set must cover). Maximal marginal relevance is related but weaker here: it avoids repeating similar passages, without knowing which parts still lack evidence.
What we measured
We tested a narrow version on our own evals: after a normal search, run one extra keyword lookup for the weaker half of a two-part question and give it one of ten result slots. The measure was complete support, meaning the top ten contains a labeled passage for every part.
- On 55 questions built by mechanically pairing single-fact questions, complete support rose from 30 to 50.
- On 24 compound questions an independent author wrote in natural phrasing, it rose from 8 to 9, with two gains and one loss. The loss was a swap that evicted the only passage supporting the other half.
- On 33 fresh compound questions, swapping only when a passage-level check showed neither half losing evidence took it from 12 to 15, with three gains and no losses.
These runs were offline, keyword-only and on one corpus of 7,086 chunks, so treat them as direction, not a product claim. The lesson we took: decomposition looks far better on questions that were assembled to be split than on questions people write, and the eviction rule matters as much as the split.
Where it breaks
- False splits. “Research and development spending in Q3” is one topic with an “and” in it. A split produces two vague searches and worse results than the original.
- Orphaned fragments. “…and how does that compare to last year?” means nothing alone. Each sub-query must carry the shared subject.
- Dependent parts. If part two needs part one’s answer, parallel searches can’t help.
- Budget pressure. Two parts share the same context budget. If every slot was already useful, giving one to the weaker half takes evidence from the stronger one.
- Cost. One search per part, plus a model call if a model does the splitting.
When to use it
Reach for decomposition on comparison questions, questions that name two documents or two entities, and checklists (“what are the notice period, the renewal term and the governing law?”). Leave it off for single lookups, where it can only add noise.
In DocSlurp
queryDecompose is off by default. It splits by pattern, requires every part to have at least three words, runs the parts in parallel and fuses them with RRF, so the caution above applies: it doesn’t reserve slots per part. In chat, the default search agent can issue one search per part itself and reads each result list separately.
In DocSlurp
In DocSlurp
queryDecompose (off by default) splits on “and”, “vs”, “compare”, “difference between” or “also” when every part has at least three words, searches the parts in parallel and fuses them with RRF (server/src/search/query-rewrite.ts, server/src/search/session-planner.ts). It reserves no slots per part and uses no model.