Skip to content

LearnRank & assemble

Sentence-window retrieval

Index single sentences for precise matching, then expand each hit to a window of the sentences around it before the model reads it.

Related techniqueGlossary entry

On this page

Sentence-window retrieval indexes single sentences and stores, with each one, the few sentences around it. Search matches the single sentence; the model reads the window.

Say the index holds the sentence “The Learjet is wheels-up within three hours of the assistant’s request.” It matches “How fast can my assistant get the Learjet to London?” sharply, because its embedding isn’t diluted by anything else. Before the passage goes into the prompt, it is swapped for its window, for example two sentences either side:

Executive assistants book the jet through the travel desk. The CFO pre-approves trips over $40k. The Learjet is wheels-up within three hours of the assistant’s request. It flies Teterboro to Farnborough in about 7h 10m. Catering and a ground car are arranged with the booking.

The sentence found it; the window explains it. LlamaIndex popularized this pattern as a node parser that stores the window in each sentence’s metadata and a post-processor that swaps it in.

The gotcha

Sentence splitting is fragile on real documents. Tables, bullet lists, headings and OCR output don’t break into clean sentences, and a fixed window can cross a section boundary into unrelated text. Sentence-level indexing also multiplies the number of vectors you store several times over compared with paragraph chunks.

DocSlurp doesn’t index single sentences. Its chunks follow Markdown structure, and the closest feature is parent retrieval, which expands a chunk hit to its page, or to the neighboring chunks when the page text is missing (core/server/src/search/session-execution.ts). See small-to-big retrieval.

In DocSlurp

Related technique

DocSlurp does not index single sentences. Its closest behavior is parent retrieval, which expands a chunk hit to its page or, as a fallback, to the neighboring chunks (core/server/src/search/session-execution.ts). See small-to-big retrieval.

Sources

Jump to