Temperature
Sampling temperature
A sampling setting that divides a model’s token scores before softmax: low values let the likeliest token dominate, high values flatten the choice.
In DocSlurpGlossary entry
On this page
Temperature controls how a model chooses among candidate next tokens. The model outputs a score (a logit) for each token; sampling divides every score by the temperature T, then applies softmax to turn scores into probabilities.
Take three candidates with logits 2, 1 and 0:
| T | Divided logits | Probabilities |
|---|---|---|
| 0.5 | 4, 2, 0 | 0.867, 0.117, 0.016 |
| 1 | 2, 1, 0 | 0.665, 0.245, 0.090 |
| 2 | 1, 0.5, 0 | 0.506, 0.307, 0.186 |
Lower T sharpens the distribution toward the top token; higher T flattens it. As T approaches 0 the top token always wins, which is why “temperature 0” is treated as greedy decoding.
For extraction, classification and grounded answers, low temperature is the usual choice: you want the same input to give the same output. For brainstorming or varied phrasings, higher temperature helps. Related settings, top-p and top-k sampling, cut off the unlikely tail instead of reshaping the whole distribution.
The gotcha
Low temperature buys consistency, not accuracy. If the model’s top token is wrong, temperature 0 repeats that wrong answer every time, which can make an error look like a fact. Even temperature 0 is not perfectly deterministic on most hosted APIs: batching and floating-point order can change results between runs. And not every endpoint honors the parameter; some ignore it silently, so check that the setting you pass is applied.
In DocSlurp
In DocSlurp
Vision extraction and output moderation set temperature 0 (core/worker/src/media/vision.ts, core/server/src/services/chat-output-moderation.ts). Chat uses the provider’s default, and /v1/chat/completions rejects a temperature parameter as unsupported (core/server/src/routes/chat-completions.ts).