Skip to content

LearnRetrieve

Vector search

Semantic search · Dense retrieval

Embeds the query and every passage as vectors, then returns the passages whose vectors point most nearly in the same direction as the query’s.

In DocSlurpIllustrated explainer

On this page
Vector search returns the passages pointing closest to the querySix passages are arrows from a common origin at 40, 55, 68, 88, 110 and 128 degrees. The question "How fast can we fly a private jet to London?" points at 52 degrees; its three nearest passages are T, travel-desk-guide page 3, cosine 0.999; A, charter-agreement page 2, cosine 0.978; and L, launch-review page 5, cosine 0.961. The off-topic question "Who won the office chili cook-off?" points at 168 degrees, where no passage is about chili, yet search still returns three: E, expansion-forecast, cosine 0.766; I, investor-call, cosine 0.530; and G, growth-plan-2027, cosine 0.174.Query“How fast can we fly a private jet to London?”Query“Who won the office chili cook-off?”ATLGIEqqonly the angle mattersTop 3 by cosine1Ttravel-desk-guide.pdf p.30.9992Acharter-agreement.pdf p.20.9783Llaunch-review.pdf p.50.9611Eexpansion-forecast.xlsx0.7662Iinvestor-call.mp40.5303Ggrowth-plan-2027.pdf p.120.174All three are about the jet.None is about chili. Still three.
Vector search returns the k passages pointing closest to the query, and it always returns k. For the jet question the three nearest arrows sit within 16° of the query. Switch to the off-topic question and the cone stretches to 80° to collect three passages anyway, the last at a cosine of 0.174. Two dimensions and illustrative angles; real embeddings have hundreds of dimensions.

The problem: the right page uses other words

Your executive assistant asks “How fast can we fly a private jet to London?” The page that answers it, in travel-desk-guide.pdf, says “the charter aircraft departs for Farnborough” within three hours of a booking. It contains none of the question’s content words: not “fast”, not “private jet”, not “London”. Keyword search scores it zero and moves on.

Vector search finds it anyway, because it compares meanings instead of spellings.

How it works: turn text into directions

An embedding model reads a piece of text and returns a list of numbers, a vector, typically 384 to 3,072 of them. The model is trained so that texts with similar meanings get vectors that point in similar directions. “Private jet to London” and “charter aircraft departs for Farnborough” end up close together; “the office chili cook-off” points somewhere else.

Search then has three steps:

  1. Index time: split every document into passages, embed each passage once, and store the vectors.
  2. Query time: embed the question with the same model.
  3. Rank: return the k stored vectors that point most nearly the same way as the query vector.

“Most nearly the same way” is usually cosine similarity: the cosine of the angle between two vectors. Same direction gives 1, a right angle gives 0. Length doesn’t matter, only direction, which is why the query arrow in the figure is shorter than the passage arrows and still ranks them correctly.

A worked example in two dimensions

Real embeddings have hundreds of dimensions, but the mechanism is the same in two, so the figure uses two. Each arrow is a passage; the angle is all that matters. The numbers are illustrative.

Passage Angle Angle from the query (52°) Cosine
T travel-desk-guide.pdf p.3 55° 3° 0.999
A charter-agreement.pdf p.2 40° 12° 0.978
L launch-review.pdf p.5 68° 16° 0.961
G growth-plan-2027.pdf p.12 88° 36° 0.809
I investor-call.mp4 110° 58° 0.530
E expansion-forecast.xlsx 128° 76° 0.242

With k = 3 the search returns T, A and L. The shaded wedge in the figure is the narrowest cone around the query that holds three passages.

Now switch the figure to the off-topic question, “Who won the office chili cook-off?”, pointing at 168°. Nothing in the corpus is about chili. Vector search still returns three passages: E at 0.766, I at 0.530 and G at 0.174. The cone just gets wider. There is always a nearest neighbor. Ranking by similarity can tell you which passages are closest; it cannot tell you whether the closest is close enough.

Exact versus approximate

Comparing the query with every stored vector is exact, and fast enough for small collections. For large ones, systems use approximate nearest neighbor indexes such as HNSW, which visit a small fraction of the vectors and occasionally miss a true neighbor in exchange for speed. The ranking idea doesn’t change; only the guarantee does.

Where it breaks

  • Rare exact strings. A tail number like N750MC, a part number or a surname carries little meaning for the model to encode. The embedding blurs it with similar-looking codes. Keyword search handles these better, which is why most systems run hybrid search.
  • Negation and numbers. “The CFO does not pre-approve this trip” and “the CFO pre-approves this trip” embed close together. “Within 3 hours” and “within 30 hours” do too.
  • No notion of “nothing found”. As the off-topic question shows, top-k always fills up. Unrelated text can still score surprisingly high: on a small test corpus, DocSlurp measured off-topic queries reaching cosines of 0.58 to 0.70 with its default local model. A relevance floor calibrated per model is the fix.
  • Questions don’t look like answers. A question can sit closer to other questions, such as a FAQ heading, than to the passage that answers it. HyDE works around this by searching with a drafted answer.
  • One model, one space. Vectors from two different models, or two versions of one model, aren’t comparable. Changing the model means re-embedding everything.

When to use it

Use vector search whenever people will describe what they want in their own words: support questions, policy lookups, “how do I” queries, anything where the corpus and the users don’t share a vocabulary. Pair it with keyword search for codes and names, fuse the two with reciprocal rank fusion, and keep a floor so that a question the corpus can’t answer comes back empty instead of confidently wrong.

In DocSlurp

In DocSlurp

The vector branch embeds the query, then runs approximate nearest-neighbor search in the project’s LanceDB table, restricted to in-scope documents before selection (core/server/src/search/lancedb.ts, core/server/src/services/search.ts). Its similarity feeds the relevance floor.

Sources

Jump to