What is a circuit breaker — and why retries make an outage worse

Here's a surprising one. You call an AI model in a loop over a hundred records. The provider's service goes down. How many requests do you send into it?
Not a hundred. Three hundred.
Because the official SDK retries on its own. In Anthropic's Python library the constant sits right there: DEFAULT_MAX_RETRIES = 2. So one call of yours is up to three trips to the server, with a pause that grows from 0.5 to 8 seconds. And it retries not only on 429 "too many requests", but on 408, 409 and every 5xx code — exactly when the service is already having a bad day.
That's what a circuit breaker is for.
What a circuit breaker is, in one line
A circuit breaker is a wrapper around a call to someone else's service that counts failures and at some point stops calling altogether.
Not "calls less often". Stops. And returns an error itself, immediately, without waiting for a timeout.
The analogy is right there in the name. The breaker in your fuse box doesn't "reduce the current" during a short — it opens the circuit entirely. Because during a short, any current is harmful. Same here: if a service is down, every request of yours is harmful twice over — it brings you no answer and adds load to them.
Why retries make an outage worse
A retry is a good idea for a random glitch. A packet got lost, the server blinked — try again and it works.
But an outage is different: the service didn't blink, it can't keep up. And now the retries stack:
- Every one of your clients tries 2–3 times.
- Every client that got no answer retries too.
- The user sees a spinner, taps the button again — and round we go.
The load doesn't stay flat, it multiplies — precisely when the service needs a breath to come back. This has a name: a retry storm. A service that would have been down for a minute stays down for ten, because nobody lets go of it.
There's a second cost people forget. While you wait on a dead service, you're holding your own connection, your thread, your memory. One broken external API drags your app under with it — even though your code is fine. That's latency turning into an outage.
Three states: closed, open, half-open
The whole pattern is a tiny machine with three positions.
Closed. Normal life: requests flow, the breaker just counts failures. The threshold is usually simple — "5 errors in a row", or "more than half the requests this minute failed".
Open. The threshold is crossed — and for, say, the next 30 seconds you call nobody. Every call returns an error or a prepared answer straight away. Zero waiting, zero load on their service, zero connections tied up on yours.
Half-open. The 30 seconds are up. You let one probe request through. It succeeds — the circuit closes and life goes on. It fails — open again for another 30 seconds.
The trick is that third state. Without it you either cut off a healthy service forever, or dump the whole queued-up backlog on it the moment it recovers — and take it down again.
How it differs from a rate limit
People mix these up constantly. The difference is simple, and it's about direction.
A rate limit is about frequency: "no more than 100 requests a minute", always, even when everything is healthy. It protects you from your own runaway loop and from someone else's abuse. How to add one by hand is in the rate limiting walkthrough.
A circuit breaker is about the state of whoever you're calling. While things are fine it doesn't interfere at all — 10,000 requests, go ahead. The moment the other side starts failing, it kills the channel entirely.
One constrains you. The other lets go of whoever is struggling. A real app has both.
What's in it for you
You're building a side project, and "fault tolerance" sounds like a phrase from a corporate deck. But here's your actual scenario: your bot calls an AI API, the provider has a five-minute incident, and for those five minutes your bot is simply dead — every request hanging in a timeout.
With a breaker it stays alive: it answers "the model is unavailable right now, try in a minute" — instantly, and with no bill for failed attempts.
The minimal version is a counter and a timestamp, about ten lines:
let fails = 0, openUntil = 0;
async function callAI(prompt) {
if (Date.now() < openUntil) throw new Error('AI unavailable');
try {
const r = await ai(prompt);
fails = 0; // success — reset
return r;
} catch (e) {
if (++fails >= 5) openUntil = Date.now() + 30_000;
throw e;
}
}
On top of that people add "what to serve instead": a cached previous result, a canned text, a cheaper model. And do write the moment it opens into your logs — otherwise you'll be hunting for why the app "silently didn't work" for exactly half a minute.
You tune the threshold and the pause by feel, but 5 failures in a row and 30 seconds are healthy starting values.
Is a circuit breaker only for big systems?
No. It's for anyone with someone else's service in a critical path: an AI API, payments, sending email. Only the implementation differs — production reaches for a library, a side project is fine with the ten lines above.
How do I know I need a breaker and not just a timeout?
A timeout limits one attempt; a breaker limits a series. If a service is down, a 10-second timeout will dutifully fire for each of a hundred requests — that's 1000 seconds of waiting for nothing. Use both: a short timeout, a breaker on top.
Reliability isn't about fancy words — it's a handful of habits like this one. In KODiQ we walk through them step by step, in plain language: from your first API call to an app that doesn't fall over with someone else's server.
The AI Architect 2027 course: step by step from computer basics to your own app and first customers. 543 short lessons on your phone.





