Glossary — Language models
Parameter
A parameter is one number inside a model, adjusted during training — large models have hundreds of billions of them.
1 tokensParameter
Simply put#
When you hear "a 70 billion parameter model", that is 70 billion numbers settled into place by training. Those numbers are everything the model "knows".
Example#
More parameters generally means a more capable model, but also a more expensive and slower one. Smaller models can beat larger ones on a narrow task.
Why it matters to you#
Parameter count is a popular marketing number, but on its own it says nothing about quality. The quality of the training data matters as much or more.