Temperature

Definition. A sampling parameter that scales the logits before the softmax, controlling how much probability mass is spread across alternatives. At temperature zero the model effectively takes the highest-probability token each step, producing near-deterministic output. Higher values flatten the distribution, increasing diversity and creativity while also increasing the chance of incoherence and fabrication.

How it works. Set temperature by task type rather than by taste. Use 0 to 0.2 for extraction, classification, code generation and anything with a schema. Use 0.4 to 0.7 for drafting and summarisation where some variation is acceptable. Use 0.8 to 1.0 only for brainstorming and creative work, and pair it with top-p sampling. Fix the seed where the provider supports it, and record temperature in your request logs so regressions can be traced.

Example. A data extraction pipeline runs at temperature 0.9 because a developer copied a creative-writing configuration. Field-level accuracy on a 500-document test set is 84 percent, with occasional invented values. Lowering to 0.0 raises accuracy to 97 percent and makes repeated runs on the same document produce identical output, which is essential because the pipeline deduplicates on output hash.

Use case. Temperature is the primary knob for trading reliability against variety. It matters most in structured extraction, where any deviation is a bug, and in ideation, where identical outputs are useless. It also affects cost indirectly, since higher temperature tends to produce longer, less disciplined output.

Warnings. Temperature zero is not a guarantee of determinism. Batching, hardware non-determinism and provider-side changes can still alter output. Never build a system that depends on byte-identical responses without caching.

Limitations. Temperature interacts with top-p and top-k, so changing one without the other produces surprising results. It cannot fix a bad prompt, and lowering it does not reduce hallucination in the way people expect: a model can be confidently wrong at temperature zero.

← Browse all glossary terms