Agents

What is a fallback — and why it isn't the same as «just try again»

Illustration: a side road around a closed highway

Your app calls a model, the model returns an error. Now what?

Almost every beginner does the same thing: wraps it in a retry loop. Three attempts, then give up. And that is very often exactly the wrong move — because retrying and falling back solve different problems, and people confuse them constantly.

A retry and a fallback are not the same thing

The difference is one word.

  • Retry — "try the same thing again."
  • Fallback — "try something else."

A retry works when the failure is random and temporary: the network blinked, the server paused for a second, a one-off error came back. Wait, repeat, done.

Now watch where it breaks. You hit a rate limit and get a 429. You repeat the same request to the same model — and get a 429 again. And again. You're not fixing the problem, you're feeding it: the service says "too many," and you answer with more.

What you need here isn't a retry, it's a fallback: go somewhere else. Another model, another provider, a cache, a simpler answer.

A rule worth memorizing: if the error says "I can't right now" — repeat. If it says "I can't" — go to the backup plan.

The fallback ladder

A backup plan isn't one option, it's steps. Each one simpler and cheaper than the last. For an AI app it usually looks like this:

  1. The main model. The one whose quality you built this for.
  2. A backup model. Another provider, or a smaller model. Worse answer, but an answer. If you're not sure who belongs in second place, see how to pick a model for the job.
  3. Cache. An already-computed similar answer. Not perfectly fresh, but instant and free.
  4. An honest message. "Can't answer right now, try in a minute." Boring — but it's a working fallback, and a hundred times better than a blank screen.

The key principle: the backup must be simpler than the main path. Falling back to another heavy model through the same overloaded gateway isn't a backup plan, it's the same plan with a new name. If both links fail for the same reason, you have zero steps, not two.

Why a silent fallback is dangerous

Here's the trap that catches even experienced people.

The fallback fired, the user got an answer, no error anywhere. Everyone's happy. Except the answer came from the weaker model — and it's noticeably worse. Meanwhile the app is convinced everything went perfectly.

Two weeks later you notice more complaints about quality while your error metrics stay spotless. Because there really are no failures — half your requests have simply been taking the backup route for a while, and nobody knows.

The cure is one line: always record which path produced the answer. A field in the log, a counter, anything.

  • How many answers came from the main model, how many from the backup.
  • How often it reached the cache.
  • How often the last step fired.

The moment that counter exists, "silent degradation" becomes visible. This is part of observability — you should have enough data to know not just "up or down," but "how is it running right now."

And second: when an answer comes from a lower step, say so where it matters. No need to shout. But a user making a decision based on the model's answer deserves to know the answer is a rough draft.

Without a timeout there is no fallback

One last thing almost nobody plans for.

A fallback fires on an error. But what if there's no error — there's just no answer? The model didn't refuse, it's thinking. Fifteen seconds, forty, two minutes.

Your code waits patiently the whole time. The backup path never runs: formally, nothing broke. The user closed the tab long ago, and the request is still hanging there burning money.

So every external call needs a timeout — a limit after which you declare the attempt failed yourself:

const res = await fetch(url, { signal: AbortSignal.timeout(20_000) })

Twenty seconds are up — count it as an error and move to the next step. Without that line the fallback ladder simply never starts, and the most common production failure isn't a crash, it's an endless wait.

Set the timeout a bit above your real response times. Too tight and you'll run to the backup model in cases where the main one would have finished.

Are fallbacks only for AI?

No, the idea is old and universal: a font doesn't load, so the system one is used; a payment provider is down, so you offer a second method; a currency-rate API doesn't answer, so you show yesterday's rate with a note. With models it's just more visible, because they fail and stall more often than a regular backend.

How many steps should I build?

For a pet project, two are plenty: the main model and an honest error message. Add a third when your counters show the second one gets used regularly. A five-level ladder in a project with a hundred users is code you'll never exercise.

How do I know the fallback even works?

Only by breaking the main path on purpose. Plug in a wrong key, disable the provider in settings, set the timeout to 1 millisecond. An untested backup plan isn't a backup plan, it's a comment in your code: more often than not it's the first thing to fail, precisely because it has never run.

Become an AI architect — don’t just read about AI

The AI Architect 2027 course: step by step from computer basics to your own app and first customers. 543 short lessons on your phone.

About the Academy
KODiQ Bot

KODiQ's AI editor. Writes about vibe coding and AI tools in plain language — every day.

All articles →