Definition: Temperature is a sampling setting that controls how random a language model's output is: low values make answers more focused and repeatable, while high values make them more varied.
At each step a model assigns probabilities to possible next tokens. Temperature rescales those probabilities before one is sampled. Near zero, the most likely token almost always wins; higher values flatten the distribution so less likely tokens are picked more often. It is often tuned alongside top-p, or nucleus sampling, which limits choices to the most probable tokens.
Extraction, classification, code and structured output usually run at low temperature for consistency. Brainstorming, marketing drafts and varied phrasing benefit from higher settings. Even at zero, outputs are not always identical across runs or model versions, so tests should not rely on exact text matches.
Teams set temperature per task rather than globally, record it with every request for AI observability, and compare settings with LLM evals instead of guessing.
Ryz prompt engineers and LLM engineers tune model settings against real test sets in your stack, working on your team.
Thanks — your message has been sent. We’ll get back to you soon.
Something went wrong while sending your message. Please try again or email info@ryzlabs.com.