Embedding space
Vector space · Latent space
The high-dimensional space a particular embedding model maps inputs into, where nearness stands for similarity and each model has its own incompatible space.
In DocSlurpIllustrated explainer
On this page
The problem: a list of numbers you can’t picture
An embedding model turns “revenue” into 384 numbers. So does “profit”, and so does “private jet”. On their own the numbers mean nothing: no position says “this is about money”. What carries meaning is how the vectors sit relative to each other. The set of all places a model can put an input is its embedding space, and learning to think about that space, without being fooled by pictures of it, is most of what you need to reason about vector search.
Nearness is meaning
The model was trained so that inputs used in similar ways land in similar directions. The result is a space with neighborhoods. Here are real cosines from all-MiniLM-L6-v2, one of DocSlurp’s local models, for single words and the question “Are we on track to beat our FY2027 earnings?”:
| From | Closest | Next | Far |
|---|---|---|---|
| revenue | profit 0.717 | income 0.670 | private jet 0.212 |
| private jet | flight 0.517 | charter 0.283 | income 0.122 |
| the question | revenue 0.317 | income 0.301 | flight 0.088 |
Finance words cluster. Travel words cluster. The question lands next to “revenue” and “income”, and as far from “flight” as anything in the set. A k-nearest neighbors search for the question would return the finance cluster first, which is the whole point.
Notice “flight” and “revenue” at 0.284, nearly as close as “revenue” and “EBITDA” at 0.282. Single words are noisy inputs: both are short business nouns that show up in similar sentence positions (a flight report, a revenue report). Full sentences carry more context and separate more cleanly.
Every picture of the space is a shadow
You can’t draw 384 dimensions, so people project them down to two with PCA, t-SNE or UMAP. The figure does this with PCA for the eight inputs above. The two axes it keeps hold 46% of the spread between the points; the other 54% is flattened away, and the picture lies in exactly those directions.
Pick “EBITDA” in the figure. On the map, the question is the closest point. In the model’s real space, it is only fourth. Meanwhile “revenue”, which looks sixth closest on the map, is really the most similar of all seven. Nothing is wrong with the projection: it is doing the best a flat picture can. The lesson is that a 2D map shows clusters reliably and individual distances unreliably.
This gets worse with the nonlinear methods. t-SNE and UMAP are tuned to keep local neighborhoods intact, and in exchange they distort the sizes of clusters and the distances between them. Two clusters drawn far apart may be close in the real space; a tight blob may be spread out. Use these maps to explore and to spot outliers, never to measure.
The geometry is strange
High-dimensional space doesn’t behave like the room you’re sitting in.
- Almost everything is nearly perpendicular. Pick two random directions in 384 dimensions and their cosine is very likely to be close to 0. Meaningful similarity shows up as a small, consistent tilt toward each other.
- Real embeddings crowd into a cone. The opposite also happens: trained models tend to put all inputs in a narrow range of directions, so even unrelated texts score well above 0. In the table, the lowest cosine is 0.088, not −1. Where “unrelated” sits varies by model, so similarity values are only comparable within one model.
- Directions can encode relations, loosely. Older word embeddings famously had “king − man + woman ≈ queen”. Modern sentence models have weaker versions of this. Don’t build features on vector arithmetic without testing it.
Each model has its own space
Two models that both output 384 numbers have unrelated coordinate systems. Dimension 17 in one has nothing to do with dimension 17 in the other. A query embedded with model A and searched against passages embedded with model B returns noise, and nothing in the arithmetic warns you, because the shapes match.
The same applies to settings within one model: pooling method, normalization, truncated dimensions and even weight quantization can move vectors enough to matter. A space is defined by the whole recipe, not the model name. That is why systems record exactly which recipe produced each vector; see embedding versioning.
Where it breaks
- Neighborhoods follow usage, not truth. “Inbound flight” and “outbound flight” appear in the same kinds of sentences, so the same model scores them 0.911, even though on an itinerary they are opposites.
- The space reflects the training data. A general model’s neighborhoods are general. In a corpus full of internal acronyms, the map may put your most important terms in the wrong place.
- Distances near the top are crowded. Many candidates sit at almost the same cosine. Small changes in wording reorder them, which is why systems fuse vector results with keyword results and rerank the shortlist.
How to use the idea
Think of vector search as “find the neighbors of the query’s position” and most behavior becomes predictable: paraphrases work because they land nearby; part numbers fail because they don’t move the vector much; switching models breaks search because the map changed. When you need to inspect a corpus, project it to 2D to find clusters and outliers, then check any distance that matters with the real cosine.
In DocSlurp
In DocSlurp
Each project keeps one vector table per embedding model and dimension count, so vectors from different spaces are never searched together (core/packages/shared/src/vector-namespace.ts). The reuse fingerprint also records provider, normalization and weights dtype (core/worker/src/embeddings/embedding-reuse.ts).