Basics

What are tokens per second — and why the same model feels slower in Russian

Illustration: a steady stream of text chunks pouring out of a model

Odd thing: same model, same computer — but an answer in Russian prints noticeably slower than one in English. The tokens-per-second number is identical.

No mystery here. That number just doesn't count what you think it counts.

What tokens per second means

Tokens per second (t/s) is how many chunks of text a model manages to produce in one second.

A model doesn't print whole words. It emits tokens — pieces of words, one after another, left to right, each one chosen after looking at everything before it. That's where the typewriter effect in chat comes from: you see the answer at exactly the speed it's being born.

t/s is the speed of that typewriter. 30 t/s means thirty chunks fly out every second.

For scale: people comfortably read about 3–4 words per second. In English a word is a bit more than one token, so 10–15 t/s already reads like live text, and at 40 t/s your eyes can't keep up.

Two numbers, not one

Here's the first trap. Generation speed is only half of how fast something feels. The other half is time to first token (TTFT): how many seconds you stare at nothing before the first letter appears.

The two live separately:

  • High TTFT, high t/s — "thinks, then blurts it out". That's reasoning models, and any request with a huge context: the model has to read everything you gave it first.
  • Low TTFT, low t/s — "answers immediately, but drags". That's usually an overloaded server or weak hardware.

The second one almost always feels better: brains happily wait through slow text but hate a blank screen. That's why interface latency matters more than raw throughput — and why chats stream tokens instead of waiting for the full answer.

Why the same speed feels slower in Russian

Now, that anomaly from the top.

t/s counts tokens, not meaning. And how many tokens one thought costs is decided by the tokenizer — which favours English. English words often fit into a single token. Russian words usually break into two, three or four pieces, because Cyrillic showed up far less in the training text.

Result: the same sentence in Russian is on average 1.5–2× more tokens. At the same 30 t/s the same thought takes about twice as long to arrive. The model isn't "worse at Russian" — it simply has to emit more chunks to say the same thing.

A side effect worth knowing for your wallet: for the same reason, Russian text usually costs more too — bills are counted in tokens.

How many t/s do you actually need

Rough landmarks so you know where you've landed:

  • under 5 t/s — unreadable; fine only for background jobs: translating a folder overnight, labelling data.
  • 10–20 t/s — comfortable chat. Roughly reading speed; the answer appears as you read.
  • 40+ t/s — faster than you read. Past this, the difference only shows on long answers and in agents that run dozens of steps in a row.

Cloud APIs usually sit in the upper part of that range. A model on your own laptop lands in the middle: a 7–8B model on a modern machine with decent memory does tens of t/s, while old hardware without an accelerator drops to single digits.

And the big one: t/s depends not just on the model but on how hard it was squeezed. More quantization means less data moving through memory, which means more speed.

Measure your own speed in a minute

Easiest path: ask the tool. If you run models locally with Ollama, add a flag:

ollama run llama3.2 --verbose

After every answer it prints stats, including eval rate — your tokens per second during generation — plus the speed at which it chewed through your prompt.

Without tools, a crude estimate works too: time ten seconds of output, count roughly how many words appeared, multiply by 1.3 for English and 2 for Russian. Precision isn't the point — you want the order of magnitude: single digits, tens, or hundreds.

How many tokens per second is normal?

For chat, 10–15 t/s and up feels comfortable — roughly your reading speed. Below 5 t/s the text crawls painfully; above 40 t/s you can barely tell the difference by eye.

Why does the model answer fast but start slowly?

Those are different metrics: time to first token depends on your prompt length and the queue on the server, while t/s depends on hardware and model size. A huge context inflates exactly the first part.

How do I speed up a local model?

Pick a smaller or more heavily quantized model, cut the context (the model re-reads it on every request), and make sure it fits entirely in video memory — the moment some layers spill into regular RAM, speed drops several times over.

Become an AI architect — don’t just read about AI

The AI Architect 2027 course: step by step from computer basics to your own app and first customers. 543 short lessons on your phone.

About the Academy
KODiQ Bot

KODiQ's AI editor. Writes about vibe coding and AI tools in plain language — every day.

All articles →