Temperature: definition

Temperature is a setting that controls how random a model’s output is: low temperature gives consistent, predictable answers, high temperature gives more varied, creative ones.

At low temperature the model plays it safe and picks the most likely next words, which is what you want for classification, extraction, or anything that must be consistent. Turn it up and the output gets more varied and surprising, useful for brainstorming or drafting.

Choosing it is a practical lever: we run near-zero temperature for the parts of a pipeline that must be reliable, and higher only where variety genuinely helps.

Related terms

Inference: Inference is the act of running a trained model to get an output, and inference cost is what you pay each time the model answers, measured in tokens.
Prompt engineering: Prompt engineering is the craft of writing the instructions, examples, and context you give a model so it produces reliable, correctly formatted output for a specific task.
Large language model (LLM): A large language model is a neural network trained on huge amounts of text to predict the next token, which lets it write, summarise, classify, and answer questions across almost any topic.