Maximal marginal relevance
MMR
Picks results one at a time, scoring each candidate by its relevance minus its similarity to what has already been picked, so near-duplicates stop crowding the list.
In DocSlurpIllustrated explainer
On this page
score = λ × relevance − (1 − λ) × max similarity to what is already picked
| Candidate | Relevance pick 1 | Pick 2 | Pick 3 | Result |
|---|---|---|---|---|
Aq3-earnings-memo.pdf p.3margin 6 points above plan | 0.900 | picked | picked | 1st |
Bboard-update-2026.pdf p.7same paragraph, copied | 0.880 | −0.035 | −0.035 | skipped: 0.95 like A |
Cgrowth-plan-2027.pdf p.12FY2027 margin plan | 0.700 | 0.200 | picked | 2nd |
Dexpansion-forecast.xlsxmargin by market | 0.580 | 0.190 | 0.140 | 3rd |
Einvestor-call.mp4 05:12CFO on margin guidance | 0.520 | 0.135 | 0.135 | not picked |
| A | B | C | D | E | |
|---|---|---|---|---|---|
| A | · | 0.95 | 0.30 | 0.20 | 0.25 |
| B | 0.95 | · | 0.30 | 0.20 | 0.25 |
| C | 0.30 | 0.30 | · | 0.30 | 0.20 |
| D | 0.20 | 0.20 | 0.30 | · | 0.15 |
| E | 0.25 | 0.25 | 0.20 | 0.15 | · |
The problem: three slots, two of them the same paragraph
Someone pasted the margin paragraph from q3-earnings-memo.pdf into board-update-2026.pdf. Now when you ask “How far above our margin target are we?”, both copies match well. Ranked by relevance, your top three are:
- A
q3-earnings-memo.pdfp.3, relevance 0.90 - B
board-update-2026.pdfp.7, the same paragraph, relevance 0.88 - C
growth-plan-2027.pdfp.12, relevance 0.70
The model reads the same sentence twice and never sees D, the expansion forecast’s margin-by-market sheet, which would have helped. B is the second most relevant passage and it adds nothing. Ranking each candidate on its own can’t notice that, because redundancy is a property of the set, not of one passage.
The fix: score what each pick adds
Maximal marginal relevance (MMR), from Carbonell and Goldstein (1998), builds the list one pick at a time. The first pick is the most relevant candidate. Every later pick maximizes:
MMR(d) = λ × relevance(d) − (1 − λ) × max similarity(d, s) over already-picked s
The first term rewards matching the question. The second term penalizes resembling anything already chosen. λ (lambda), between 0 and 1, sets the trade-off. Similarity is usually cosine similarity between the candidates’ embeddings.
Worked example
Five candidates with illustrative relevance scores and pairwise similarities:
| A | B | C | D | E | |
|---|---|---|---|---|---|
| Relevance | 0.90 | 0.88 | 0.70 | 0.58 | 0.52 |
| Similarity to A | 0.95 | 0.30 | 0.20 | 0.25 | |
| Similarity to C | 0.30 | 0.30 | 0.30 | 0.20 |
Pick three with λ = 0.5.
Pick 1: A, the most relevant.
Pick 2: score each remaining candidate against A.
| Candidate | 0.5 × relevance | 0.5 × similarity to A | MMR |
|---|---|---|---|
| B | 0.440 | 0.475 | −0.035 |
| C | 0.350 | 0.150 | 0.200 |
| D | 0.290 | 0.100 | 0.190 |
| E | 0.260 | 0.125 | 0.135 |
C wins. B, the near-copy, drops to last, below zero.
Pick 3: the penalty now uses the larger of each candidate’s similarities to A and to C.
| Candidate | 0.5 × relevance | 0.5 × max similarity | MMR |
|---|---|---|---|
| D | 0.290 | 0.5 × max(0.20, 0.30) = 0.150 | 0.140 |
| E | 0.260 | 0.5 × max(0.25, 0.20) = 0.125 | 0.135 |
| B | 0.440 | 0.5 × max(0.95, 0.30) = 0.475 | −0.035 |
D wins narrowly. The final list is A, C, D. The model gets the Q3 beat, the FY2027 margin plan and the margin by market, instead of the Q3 beat twice.
What λ does
Drag the slider in the figure to watch the picks change:
- λ = 1.0 to 0.8: A, B, C. With no diversity weight, MMR is plain relevance ranking, duplicate included.
- λ = 0.7: A, C, B. The copy is demoted to third.
- λ = 0.6 down to 0.3: B is gone. You get A, C, D at λ = 0.5 and 0.6, and A, D, C at 0.3 and 0.4.
- λ = 0.2 to 0.0: A, D, E. Relevance barely counts. C, the best remaining passage, loses to D and E because it is slightly more similar to A.
Low λ fails differently from high λ: it trades relevant passages for merely different ones. No λ is right for every corpus, so tune it on your own queries.
Where it breaks
- The two terms must share a scale. The formula subtracts a similarity in 0 to 1 from a relevance score. If relevance is a BM25 score of 14.2, λ = 0.5 means nothing. Normalize relevance first (see score normalization).
- It needs a similarity between candidates. Embeddings are the usual choice, but any passage similarity works. The original paper compared passages by their words.
- Corroboration looks like redundancy. Two independent documents stating the same fact are near-duplicates to MMR. For citations, a second source can be worth the slot.
- It is greedy. Each pick is locally best. That is usually fine, but it does not find the best possible set.
- It doesn’t make weak evidence stronger. It only chooses among what retrieval returned.
In DocSlurp
The algorithm is in core/server/src/search/mmr.ts. Search and chat accept an mmr flag and an mmrLambda (project default 0.4), and MMR is off by default.
Fused candidates don’t carry their embeddings, so DocSlurp compares them by their words: each passage becomes a term-frequency vector, and similarity is the cosine between those. This is the measure from the original paper. It costs no extra storage read and also works for passages only keyword search found. Relevance is rescaled to 0 to 1 before the subtraction, because fused scores sit in a narrow band near 0.03. With MMR on, each retriever returns twice as many candidates, so there are alternatives to choose from. The search diagnostics record an mmr entry showing which similarity was used and how many results MMR changed. Exact duplicates are handled earlier: see result deduplication.
In DocSlurp
In DocSlurp
Opt in with mmr and mmrLambda on search and chat (core/server/src/search/mmr.ts). Candidates are compared by term-frequency cosine, with relevance rescaled to 0 to 1, and the search diagnostics report how many results MMR changed.