Sparse vectors
Learned sparse retrieval · SPLADE
Vectors with one slot per vocabulary term where almost every value is zero; the few nonzero weights say which words matter and how much.
Related techniqueGlossary entry
On this page
A sparse vector has one slot for every term in a vocabulary, tens of thousands of slots, and almost all of them are zero. The few nonzero values say which terms the text contains and how much each matters. A dense embedding is the opposite: a few hundred slots, all filled, none individually meaningful.
Classic keyword search is already sparse-vector search. BM25 gives each query term a weight from its frequency and rarity, and a passage’s score is effectively the dot product of two sparse vectors. Because only the nonzero slots matter, an inverted index can find matches without touching most documents.
Learned sparse models such as SPLADE keep the sparse format but let a transformer choose the weights, including terms that never appear in the text. Illustratively, the passage “Book the charter aircraft for London” might get weights like charter: 2.1, aircraft: 1.8, london: 1.5 plus expansion terms jet: 0.6, flight: 0.4. A query for “jet” now matches it through the inverted index, with no dense vectors involved. This narrows the vocabulary mismatch gap while keeping exact-term behavior and explainable scores.
The gotchas: learned sparse vectors depend on the model’s vocabulary and tokenizer, so they share the weaknesses of tokenization on codes and numbers. Expansion terms can drift off-topic. And “sparse” doesn’t mean small: expansion makes postings lists longer, so indexes grow larger than plain BM25. Many systems instead pair plain BM25 with dense vectors in hybrid search.
In DocSlurp
Related technique
DocSlurp has no learned sparse vectors. Its keyword side is BM25 over LanceDB’s native full-text index (core/server/src/search/lancedb.ts), which is sparse retrieval with fixed weights. BGE-M3 can emit sparse weights, but DocSlurp uses only its dense output.