Articles
Estonian costs more in AI — and here is the proof
The same meaning, three languages, three very different bills. We measured it ourselves: Estonian burns about 1.44 times more tokens than English.
If you have ever felt that AI services hit their limits faster in Estonian, it is not your imagination. It is measurable.
The test#
We took one and the same meaning in three languages — same content, same length, only the language differing — and ran it through o200k_base, the encoding GPT-4o and GPT-5 use.
| Language | Tokens | Characters per token | Ratio |
|---|---|---|---|
| Eesti | 65 | 3.26 | 1.44× |
| English | 45 | 5.00 | 1.00× |
| Русский | 48 | 4.40 | 1.07× |
Same content. Three very different bills.
Why this happens#
A tokenizer is not neutral between languages. It was trained on a corpus containing vastly more English than all other languages combined. Frequent English words earned their own token. Estonian words, seen rarely in training, get chopped into pieces.
Estonian is also a language of compounds and cases. "Kontekstiaknast" is not one thing to a machine — it is several pieces, reassembled every time.
What it means in practice#
The bill is bigger. If your application processes Estonian text, the same work costs roughly 40–45% more tokens than it would in English. Same model, same task, larger invoice.
The context window fills sooner. A 128,000-token window holds about a third less Estonian text. A long document that fits in English may not fit in Estonian.
Answers come out shorter. When output length is capped in tokens, you get less content in Estonian than in English.
And Russian?#
Here is the surprise: Russian does not suffer the same problem. In older encodings Cyrillic genuinely was expensive, but o200k_base handles it well — 1.07× against English, essentially level.
What to do about it#
If cost matters, a few practical moves: keep the system prompt in English even when the conversation is in Estonian; do not make the model needlessly repeat long Estonian passages back to you; and when processing large volumes of documents, budget that extra 40%.
The numbers in this article are not copied from anywhere — they are computed every time this page is built. You can repeat the experiment yourself in the tokenizer.
