Basics

What is a circuit breaker — and why retries make an outage worse

Illustration: the circuit is open — requests no longer fly at the service that's down

Here's a surprising one. You call an AI model in a loop over a hundred records. The provider's service goes down. How many requests do you send into it?

Not a hundred. Three hundred.

Because the official SDK retries on its own. In Anthropic's Python library the constant sits right there: DEFAULT_MAX_RETRIES = 2. So one call of yours is up to three trips to the server, with a pause that grows from 0.5 to 8 seconds. And it retries not only on 429 "too many requests", but on 408, 409 and every 5xx code — exactly when the service is already having a bad day.

That's what a circuit breaker is for.

What a circuit breaker is, in one line

A circuit breaker is a wrapper around a call to someone else's service that counts failures and at some point stops calling altogether.

Not "calls less often". Stops. And returns an error itself, immediately, without waiting for a timeout.

The analogy is right there in the name. The breaker in your fuse box doesn't "reduce the current" during a short — it opens the circuit entirely. Because during a short, any current is harmful. Same here: if a service is down, every request of yours is harmful twice over — it brings you no answer and adds load to them.

Why retries make an outage worse

A retry is a good idea for a random glitch. A packet got lost, the server blinked — try again and it works.

But an outage is different: the service didn't blink, it can't keep up. And now the retries stack:

  • Every one of your clients tries 2–3 times.
  • Every client that got no answer retries too.
  • The user sees a spinner, taps the button again — and round we go.

The load doesn't stay flat, it multiplies — precisely when the service needs a breath to come back. This has a name: a retry storm. A service that would have been down for a minute stays down for ten, because nobody lets go of it.

There's a second cost people forget. While you wait on a dead service, you're holding your own connection, your thread, your memory. One broken external API drags your app under with it — even though your code is fine. That's latency turning into an outage.

Three states: closed, open, half-open

The whole pattern is a tiny machine with three positions.

Closed. Normal life: requests flow, the breaker just counts failures. The threshold is usually simple — "5 errors in a row", or "more than half the requests this minute failed".

Open. The threshold is crossed — and for, say, the next 30 seconds you call nobody. Every call returns an error or a prepared answer straight away. Zero waiting, zero load on their service, zero connections tied up on yours.

Half-open. The 30 seconds are up. You let one probe request through. It succeeds — the circuit closes and life goes on. It fails — open again for another 30 seconds.

The trick is that third state. Without it you either cut off a healthy service forever, or dump the whole queued-up backlog on it the moment it recovers — and take it down again.

How it differs from a rate limit

People mix these up constantly. The difference is simple, and it's about direction.

A rate limit is about frequency: "no more than 100 requests a minute", always, even when everything is healthy. It protects you from your own runaway loop and from someone else's abuse. How to add one by hand is in the rate limiting walkthrough.

A circuit breaker is about the state of whoever you're calling. While things are fine it doesn't interfere at all — 10,000 requests, go ahead. The moment the other side starts failing, it kills the channel entirely.

One constrains you. The other lets go of whoever is struggling. A real app has both.

What's in it for you

You're building a side project, and "fault tolerance" sounds like a phrase from a corporate deck. But here's your actual scenario: your bot calls an AI API, the provider has a five-minute incident, and for those five minutes your bot is simply dead — every request hanging in a timeout.

With a breaker it stays alive: it answers "the model is unavailable right now, try in a minute" — instantly, and with no bill for failed attempts.

The minimal version is a counter and a timestamp, about ten lines:

let fails = 0, openUntil = 0;

async function callAI(prompt) {
  if (Date.now() < openUntil) throw new Error('AI unavailable');
  try {
    const r = await ai(prompt);
    fails = 0;              // success — reset
    return r;
  } catch (e) {
    if (++fails >= 5) openUntil = Date.now() + 30_000;
    throw e;
  }
}

On top of that people add "what to serve instead": a cached previous result, a canned text, a cheaper model. And do write the moment it opens into your logs — otherwise you'll be hunting for why the app "silently didn't work" for exactly half a minute.

You tune the threshold and the pause by feel, but 5 failures in a row and 30 seconds are healthy starting values.

Is a circuit breaker only for big systems?

No. It's for anyone with someone else's service in a critical path: an AI API, payments, sending email. Only the implementation differs — production reaches for a library, a side project is fine with the ten lines above.

How do I know I need a breaker and not just a timeout?

A timeout limits one attempt; a breaker limits a series. If a service is down, a 10-second timeout will dutifully fire for each of a hundred requests — that's 1000 seconds of waiting for nothing. Use both: a short timeout, a breaker on top.

Reliability isn't about fancy words — it's a handful of habits like this one. In KODiQ we walk through them step by step, in plain language: from your first API call to an app that doesn't fall over with someone else's server.

Become an AI architect — don’t just read about AI

The AI Architect 2027 course: step by step from computer basics to your own app and first customers. 543 short lessons on your phone.

About the Academy
KODiQ Bot

KODiQ's AI editor. Writes about vibe coding and AI tools in plain language — every day.

All articles →