Map-reduce summarization
Hierarchical summarization
Summarizing text too long for one model call by summarizing each piece separately (map), then combining the partial summaries into one (reduce).
Related techniqueGlossary entry
On this page
Map-reduce summarization handles text too long for one model call. Split it into pieces that fit, summarize each piece on its own (the map step, which can run in parallel), then summarize the summaries (the reduce step). If the partial summaries are still too long together, reduce again, building a tree.
Summarize all eight documents of the finance corpus: map produces one paragraph each for growth-plan-2027.pdf, q3-earnings-memo.pdf, board-update-2026.pdf and the rest; reduce merges eight paragraphs into one page. The same pattern answers a question across a corpus: map asks each piece “what does this say about FY2027 earnings?”, reduce combines the answers. Microsoft’s GraphRAG global search does this over community summaries, scoring each partial answer for helpfulness and dropping the useless ones before the reduce.
The gotcha
Information only flows upward. The reduce step sees nothing the map step left out, so a detail one map call judged minor (“EBITDA guidance raised from $9.1M to $10.6M”) can never reach the final summary, even if it matters in context. Contradictions between pieces also get smoothed into vague agreement. Asking map calls to keep numbers, names and disagreements verbatim reduces the loss.
In DocSlurp
DocSlurp does not use map-reduce. A document summary is one model call. For a long document, the input is a sample of at most 10 pages (the first three, the last, and one from each stretch between), clipped to about 10 KB of text and labeled as sampled so the model doesn’t claim to have read everything (core/worker/src/stages/summary-evidence.ts). That trades completeness for one bounded call per document.
In DocSlurp
Related technique
DocSlurp does not use map-reduce. A document summary is one model call; long documents are represented by a labeled sample of at most 10 pages (core/worker/src/stages/summary-evidence.ts). Community summaries are written per community but never reduced into a global answer.