What is top-p — and why the newest models ripped that knob out

Here's the irony. Every API guide tells you about top-p: "tune top-p and the model gets sharper." And in Anthropic's newest models — Opus 4.8, Sonnet 5 — the knob is simply gone. Send top_p in your request and you get a 400 error instead of an answer.

The knob everyone tells you to turn has been ripped out. Let's look at what it did — and why it's no great loss.

What top-p means in plain words

The model doesn't pick the next word. It lays every word out by probability: "house" — 40%, "haze" — 12%, "dragon" — 0.003%. And so on, all the way down to the bottom of the vocabulary.

Then something has to be pulled from that stack. Draw honestly by probability and "dragon" comes up eventually — rarely, but it comes up. One hit like that wrecks the whole answer.

top-p (fancy name: nucleus sampling) fixes this bluntly and effectively: it cuts off the tail.

Here's how. Add probabilities from the top down until the sum reaches p. Everything that didn't fit gets thrown away. At top_p = 0.9 the model keeps the smallest set of options that adds up to 90% of the probability, and draws only from that. "Dragon" doesn't make the cut.

The edges are worth memorizing:

• top_p = 1.0 — nothing is cut. Every word is in play, bottom of the barrel included.
• top_p = 0.9 — the working classic. Tail trimmed, variety alive.
• top_p = 0.1 — only the very top survives. Answers get predictable to the point of boredom.

Notice: the size of the surviving set floats. Where the model is confident, one word hits 90% on its own. Where it hesitates, it takes twenty. That's the whole trick: the threshold is on probability, not on a count.

How top-p differs from temperature

Both knobs reach into the same stack of probabilities, and they do different things.

Temperature reshapes the stack. Low — the gap between the leader and the rest grows, the favorite wins nearly always. High — probabilities flatten out and the underdogs get a shot. Temperature removes nobody; it re-weights.

top-p trims the stack. It leaves the shape alone and throws out the tail.

In practice: temperature pushes on the whole answer, top-p only removes the rare nonsense. Want lively text without a surprise "dragon"? That's top-p's job, not temperature's.

Order matters too: temperature is applied first, then top-p trims the already-reshaped distribution.

Why you can't turn both at once

This is the advice that should have come first: change one knob, leave the other at its default.

Not because of magic — because the effects multiply into something unreadable. High temperature plus low top-p: the model got loose, then everything but the top got sliced off, so you get ordinary words in a strange order. Low temperature plus high top-p: there's nothing left to trim, the distribution already collapsed onto one favorite.

Either way, you won't know which knob produced the result. And you won't reproduce it.

So: turning temperature? Leave top_p = 1.0. Turning top_p? Leave temperature at default. That's exactly what every provider's documentation has been saying for years.

What if the knobs are gone

Which brings back the twist from the intro. Anthropic's newest models (Opus 4.8, Opus 4.7, Sonnet 5) don't ignore temperature, top_p, and top_k — they removed them. A request carrying them fails with a 400 invalid_request_error. Not "the model didn't notice," but "the model refuses to talk."

This isn't a whim. The logic is simple: sampling is too blunt a tool for what people were actually reaching for.

Set temperature to zero for stability? It never guaranteed identical answers anyway — why AI gives different answers — so the guarantee was imaginary. Raised temperature for creativity? Say so in words: "give me four distinct options that don't resemble each other" beats any slider.

So the new lay of the land:

• Models that still have the knobs (most, OpenAI included) — the one-at-a-time rule holds.
• Anthropic's newest models — no knobs; you steer behavior with the prompt.

Which is probably the best possible outcome for a beginner. While everyone argued about what to tune, the answer turned out to be "explain the task properly."

What top-p value should I use by default?

None — leave it at 1.0 (i.e. don't touch it). The knob earns its keep only when the model occasionally drops in an out-of-place rare word. Then try 0.9. If answers go bland, put it back.

Are top-p and top-k the same thing?

No. top-k keeps a fixed number of options: "exactly 40, cut the rest." top-p keeps however many it takes to reach a given probability — so it adapts to how confident the model is. That's why top-p usually behaves more sensibly.