Skip to content

LearnRepresent

Matryoshka embeddings

Matryoshka representation learning · MRL

Embeddings trained so that the first k values are themselves a usable, smaller embedding, letting you cut a vector short to save storage and compute.

In DocSlurpGlossary entry

On this page

A Matryoshka embedding is trained so that its first few values work as a smaller embedding on their own. Take a 1,024-dimension vector, keep the first 256 values, re-normalize, and you have a usable 256-dimension embedding: four times smaller, somewhat less precise. The name comes from Russian nesting dolls: each smaller vector sits inside the larger one.

It works because training scores the model at several sizes at once (say 64, 128, 256, 512 and the full width), so the model learns to put the most important information first. An ordinary embedding spreads information evenly, and truncating it produces garbage.

Truncation changes the vector’s length, so it must be re-normalized:

full:       (0.6, 0.48, 0.64)    length 1
first two:  (0.6, 0.48)          length 0.768
rescaled:   (0.781, 0.625)       length 1

The common pattern is a two-stage search: find candidates quickly with short vectors, then rescore the shortlist with the full ones. OpenAI’s text-embedding-3 models expose this through a dimensions parameter.

The gotchas: only truncate models trained this way; check the model card. Truncated and full vectors live in different embedding spaces for search purposes, so never mix sizes in one index. And re-normalize after cutting, or cosine and dot product stop agreeing.

In DocSlurp

In DocSlurp

Custom dimensions are accepted for OpenAI models that support them, local Transformers.js models and local vLLM, Ollama or LM Studio servers. Truncation keeps the first k values and re-normalizes (fitEmbeddingDimensions in core/packages/shared/src/embedding.ts). Hosted Perplexity and BGE-M3 routes reject resizing.

Sources

Jump to