Skip to content

LearnRepresent

Dot product

Inner product

Multiply two vectors position by position and add the results; the single number grows when the vectors point the same way and are long.

Background conceptGlossary entry

On this page

The dot product of two vectors multiplies them position by position and adds the results:

a = (2, 1, 0)    b = (1, 3, 2)
a · b = 2×1 + 1×3 + 0×2 = 5

Geometrically, a · b = |a| × |b| × cos θ, where θ is the angle between the arrows. So the dot product is large when the vectors point the same way and when they are long. It is zero when they are perpendicular and negative when they point apart.

That dependence on length is the gotcha. Compare a query q = (1, 0) with two passages:

Passage Dot product with q Cosine with q
(0.9, 0.1) 0.9 0.994
(2, 2) 2.0 0.707

The dot product prefers the long vector at a 45° angle; cosine similarity prefers the short one that points almost exactly at the query. Which is right depends on how the model was trained. Some models deliberately encode extra signal in length and expect a dot product; most text embedding models expect cosine.

The usual resolution is normalization: scale every vector to length 1 when storing it. Then |a| = |b| = 1, the dot product equals the cosine, and you get the cheapest calculation with the length-independent meaning.

Dot products are also the core operation inside transformers: attention scores are dot products between token vectors, and a late-interaction model like ColBERT sums the best dot product for each query token.

In DocSlurp

Background concept

Dot products are the core of every similarity DocSlurp computes. Embeddings are unit length, so a dot product equals cosine; MMR computes the full cosine formula anyway (core/server/src/search/mmr.ts).

Sources

Jump to