Contextual retrieval
Contextual chunk embeddings · Chunk preambles
Prefixing each chunk with a short description of where it comes from before embedding and indexing it, so a chunk that says "it" still knows what "it" is.
In DocSlurpGlossary entry
On this page
Contextual retrieval prepends a short description of a chunk’s source to the chunk before embedding and indexing it. The problem it fixes: chunks are cut out of their documents, and many make no sense alone.
Page 12 of growth-plan-2027.pdf might be chunked into: “Projected at $48.2M, up 31% year over year and 12% ahead of plan.” Projected for what? Which year? A question about “the FY2027 revenue forecast” may miss it, because nothing in the chunk says “FY2027” or “revenue”. With a preamble, the text that gets embedded becomes:
Document: growth-plan-2027.pdf
Context: Meridian & Co.'s FY2027 strategic plan and earnings outlook.
Section: Earnings outlook > Revenue forecast
Pages: 12
Projected at $48.2M, up 31% year over year and 12% ahead of plan.
Now the chunk carries its topic, its section and its location, both for vector search and for keyword search if the preamble is indexed there too. Anthropic’s version of the technique uses a language model to write a custom context sentence for each chunk. A cheaper variant writes one summary per document and reuses it for every chunk, plus the heading path, which needs one model call per document instead of one per chunk.
A related approach, late chunking, embeds the whole document with a long-context model first and pools token vectors per chunk afterwards, so each chunk vector absorbs context without any added text.
The gotchas: a preamble shared by every chunk of a document pulls all of them toward the same direction, which can blur the differences between chunks. Changing the preamble format changes every vector, so it is part of the embedding version. And an LLM-written context can be wrong; treat it as a retrieval aid, never as evidence to cite.
In DocSlurp
In DocSlurp
On by default (contextualChunkPreambles): each chunk is embedded with a preamble of document title, a one-sentence document context, section path and pages (core/worker/src/stages/embedding-runtime.ts). The context is one summary per document, not one per chunk, and falls back to the first sentence when no hosted model is allowed.