Skip to content

LearnEvaluate

nDCG

Normalized discounted cumulative gain · nDCG@k

A ranking score that rewards highly relevant results near the top: each result’s graded gain is divided by log2(position + 1), summed, and compared with the best possible order.

Background conceptIllustrated explainer

On this page
nDCG worked by hand for five graded resultsFive results with relevance grades: growth-plan-2027.pdf page 12, grade 3; q3-earnings-memo.pdf page 3, grade 2; board-update-2026.pdf page 7, grade 2; launch-review.pdf page 2, grade 1; travel-desk-guide.pdf page 4, grade 0. Position discounts, 1 divided by log2 of position plus 1, are 1.000, 0.631, 0.500, 0.431 and 0.387. Our ranking is q3-earnings-memo, growth-plan-2027, travel-desk-guide, board-update-2026, launch-review. Their discounted gains are 2.000, 1.893, 0.000, 0.861 and 0.387, so DCG = 5.141. The ideal order, growth-plan-2027, q3-earnings-memo, board-update-2026, launch-review, travel-desk-guide, gives 3.000, 1.262, 1.000, 0.431 and 0.000, so ideal DCG = 5.693. nDCG = 5.141 / 5.693 = 0.903. Swapping only the top two results raises DCG to 5.510 and nDCG to 0.968.resultand gradegainand worth× 1 / log2(p+1)0123p1 × 1.000p2 × 0.631p3 × 0.500p4 × 0.431p5 × 0.387DCGideal5.693q3-earnings-memop.3grade 22.000growth-plan-2027p.12grade 31.893travel-desk-guidep.4grade 00.000board-update-2026p.7grade 20.861launch-reviewp.2grade 10.3875.141nDCG@5 = 5.141 / 5.693 = 0.903growth-plan-2027p.12grade 33.000q3-earnings-memop.3grade 21.262travel-desk-guidep.4grade 00.000board-update-2026p.7grade 20.861launch-reviewp.2grade 10.3875.510nDCG@5 = 5.510 / 5.693 = 0.968growth-plan-2027p.12grade 33.000q3-earnings-memop.3grade 21.262board-update-2026p.7grade 21.000launch-reviewp.2grade 10.431travel-desk-guidep.4grade 00.0005.693nDCG@5 = 5.693 / 5.693 = 1.000
The best passage is worth 3 at rank 1 and only 1.893 at rank 2. Each outline bar is a result’s grade; the filled part is what survives the position discount. Our ranking scores DCG 5.141 against an ideal 5.693, so nDCG = 0.903. Swapping just the top two recovers most of the gap (0.968), because the discount is steepest at the top.

The problem: two rankings, same recall

Two retrievers return the same five passages for “Are we on track to beat our FY2027 earnings?” One puts the passage that answers it at rank 1. The other puts it at rank 2, behind a page that is merely related. Recall@5 is identical for both: the same passages made the top five. But the first ranking is better, because the model, like a person, gives the top of the list the most weight.

You need a metric that cares about two things recall ignores: how relevant each result is, and where it sits.

Graded gains

Start with graded relevance judgments instead of yes or no. Here, on a 0 to 3 scale:

Result Grade Why
growth-plan-2027.pdf p. 12 3 states the FY2027 projection directly
q3-earnings-memo.pdf p. 3 2 shows Q3 beating plan
board-update-2026.pdf p. 7 2 gives the raised EBITDA guidance
launch-review.pdf p. 2 1 related context
travel-desk-guide.pdf p. 4 0 not relevant

The grade is the result’s gain: what a reader gets from it.

Discounting by position

Gains lower in the list are worth less, because fewer readers (and less model attention) reach them. Discounted cumulative gain divides each gain by log2(position + 1) and adds them up:

DCG@k = Σ over positions p = 1..k   gain(p) / log2(p + 1)

The discounts for positions 1 to 5 are 1.000, 0.631, 0.500, 0.431 and 0.387. The drop is steep at the top and flattens later: moving from rank 1 to 2 costs a result 37% of its value; moving from rank 4 to 5 costs it another 4%.

Our ranking puts the results in this order: memo (2), plan (3), guide (0), board update (2), review (1).

DCG = 2×1.000 + 3×0.631 + 0×0.500 + 2×0.431 + 1×0.387
    = 2.000 + 1.893 + 0.000 + 0.861 + 0.387
    = 5.141

Normalizing

5.141 means nothing alone: a query with more relevant passages can score higher just by having them. So divide by the DCG of the ideal order, the same results sorted by grade (plan, memo, board update, review, guide):

IDCG = 3.000 + 1.262 + 1.000 + 0.431 + 0.000 = 5.693
nDCG@5 = 5.141 / 5.693 = 0.903

nDCG is always between 0 and 1, and 1 means “no better order exists for these judgments”. That makes it comparable across queries, so you can average it.

The figure lets you swap the top two results. That single change, putting the grade-3 page first, raises DCG to 5.510 and nDCG to 0.968: most of the missing points were lost at rank 1. Placing the irrelevant guide at rank 3 costs less, because the discount there is already smaller.

Two common variants

  • Exponential gain. Some implementations, including DocSlurp’s embedding comparison script (core/worker/scripts/eval-bge-m3-pooling.ts), use 2^grade − 1 instead of the grade. That widens the gap between grades: a 3 is worth 7, a 2 is worth 3. With exponential gains the same ranking scores 0.840 instead of 0.903, because misplacing the top-graded page costs more.
  • Cutoff k. nDCG@10 is the common default for search benchmarks. With binary grades (0 or 1), nDCG behaves like a smoothed, position-aware recall.

Always report which variant you used; scores from different variants aren’t comparable.

Where it breaks

  • It needs graded labels. Grading is slower and less consistent than yes/no, and different assessors disagree on 2 versus 3. With only binary labels, much of nDCG’s advantage disappears.
  • The ideal is computed from what you judged. Unjudged results count as 0, and the ideal order only includes judged passages. A system that surfaces a great unjudged passage is penalized for it.
  • It normalizes per query, then averages. A query with one relevant passage and a query with twenty count equally. That is usually what you want, but it can hide that a few hard queries are failing completely.
  • The log discount is a convention. It models a reader who scans down the list. A model reading all ten passages at once may not discount that way; lost in the middle effects even penalize the middle more than the end.

When to use it

Use nDCG when ranking quality matters inside the cutoff, especially when comparing rerankers or fusion settings that reorder the same candidates. Pair it with recall@k at a deeper cutoff, which tells you whether the candidates were there to reorder.

DocSlurp uses nDCG@10 as one of the quality gates for offline search tuning, together with recall@5, recall@10 and MRR, so a latency optimization can’t silently change result quality (core/docs/evals/search-latency-tuning-2026-10-23.md).

In DocSlurp

Background concept

DocSlurp uses nDCG@10 as a quality gate in offline search tuning, next to recall@5, recall@10 and MRR (core/docs/evals/search-latency-tuning-2026-10-23.md), and in its embedding comparison script (core/worker/scripts/eval-bge-m3-pooling.ts).

Sources

Jump to