Glossary — Language models
Temperature
Temperature is the setting that controls how boldly a model picks while predicting — low gives predictable answers, high gives more creative and more random ones.
1 tokensTemperature
Simply put#
At every step the model has a set of possible next tokens with probabilities attached. Low temperature almost always takes the most likely one. High temperature sometimes takes a less likely one.
Example#
Extracting data from a document wants low temperature — the same input should always give the same output. Brainstorming wants it higher.
Why it matters to you#
You do not see this in a normal chat interface, but through an API it is one of the most important dials. If your AI feature gives unstable results, check temperature first.