Skip to content

LearnRepresent

Dimensionality

Embedding dimensions

The number of values in each embedding vector, typically a few hundred to a few thousand, which sets storage size and search cost.

In DocSlurpGlossary entry

On this page

Dimensionality is how many numbers each embedding holds. Common sizes are 384 (all-MiniLM-L6-v2), 768, 1,024 (BGE-M3) and 1,536 to 3,072 for large hosted models. It is fixed by the model: every input, a three-word query or a full page, comes out the same size.

More dimensions give the model more room to separate fine distinctions, and they cost more in a direct, predictable way. Stored as 32-bit floats, each dimension takes 4 bytes:

Dimensions Bytes per vector 1 million chunks
384 1,536 1.5 GB
1,024 4,096 4.1 GB
2,560 10,240 10.2 GB

Search work grows the same way, since every comparison touches every dimension. That’s why quantization and Matryoshka truncation exist: to keep most of the quality at a fraction of the size.

Bigger is not automatically better. A well-trained 384-dimension model can beat a poorly matched 1,536-dimension one on your corpus. Compare models on your own queries, not on dimension count.

The gotchas: dimensions are not features, so you can’t drop “unimportant” ones from an ordinary embedding without breaking it. And the “curse of dimensionality” is real but often misread: in high dimensions, random vectors are nearly perpendicular and distances bunch together. Embeddings work anyway because real data sits on a much smaller structure inside the space, but it is why similarity scores cluster in a narrow band and small differences carry the signal.

In DocSlurp

In DocSlurp

The default hosted profile returns 1,024 dimensions; catalog models range from 384 to 2,560 (core/packages/shared/src/embedding.ts). Vectors live in a namespace per project, model and dimension count (core/packages/shared/src/vector-namespace.ts).

Sources

Jump to