What is a timeout — and why your request breaks somewhere else than you think

Classic scene. Your app asks the model for something big. The response dies at exactly thirty seconds.
You search, find "raise your function limit," raise it to five minutes — and it still dies at thirty. Magic? No. You raised the wrong timeout.
A timeout is a promise, not a bug
A timeout is a declared limit on waiting: "I'll wait this many seconds for an answer, then I hang up."
Sounds petty, but without it everything jams. A connection that waits forever holds memory and a slot, and a couple hundred stuck requests take the whole server down. A timeout is protection against silent hanging: an honest error after a minute beats a tab spinning until the end of time.
The key thing to get straight: a timeout isn't about speed, it's about patience. A slow answer is latency. A cut-off answer is somebody's patience running out.
There are several, and the shortest one wins
Here's what few people hold in mind: a request travels through a chain, and every link has its own timeout.
A typical chain for an AI request:
- The browser / your
fetch— how long the frontend waits. - A CDN or proxy in front of the site — Cloudflare, nginx.
- The server or serverless function — the limit you went to bump.
- The model's SDK — client libraries ship their own default.
- The database — its own per-query limit.
The break always comes from the most impatient link. Raise the limit on number four while number two is the one cutting you off, and nothing changes.
A telling example is the "10-second limit on free hosting." That advice has been circling forums since 2023 and is long stale: Vercel today defaults to 300 seconds even on Hobby, and paid plans go up to 800 seconds, or 30 minutes in beta. So if you're dying at thirty seconds, it definitely isn't the host.
Finding the culprit in two minutes
Good news: the break is almost always signed. Look at two things.
The second it dies. If it's stable and round, that's a layer's signature. Always exactly 30, 60 or 100? Then a limit is set, and you just have to remember whose.
- ~100 seconds and an error page saying 524 — that's Cloudflare: it waited for your server and gave up.
- Exactly 60 seconds on your own server — almost certainly nginx with default settings.
- A wobbly time with no round number — that's not a timeout, that's a crash or a dropped connection.
Who sent the error. A polished Cloudflare error page — Cloudflare cut you off. ETIMEDOUT or AbortError in the browser console — the client did. A "task timed out" line in the function logs — the host did. Every layer complains in its own voice.
Then apply the rule that saves hours: fix the shortest link, not the loudest one. Usually raising the limit there is enough, and everything else can stay as it is.
When the job really is long
Sometimes it isn't a setting: the work genuinely takes three minutes. Cranking every timeout in the chain is the wrong path — the user stares at a spinner the whole time, and one dropped connection wipes the result.
Two sane options.
Streaming. The answer arrives in chunks over a live connection, and the first text shows up in about a second. It also defuses half of the speed complaints — we cover how separately.
A queue. The app answers immediately with "job accepted, here's its id," the work moves to the background, and the client polls for the result or gets a webhook. That's what everyone generating video or big reports does: minutes of waiting are normal there, and holding an HTTP connection open is pointless. It matters even more on serverless, where a long function is also billed by the second.
What to walk away with: a timeout isn't a breakage, it's someone's patience running out first. Find the most impatient link by the round second and the voice of the error — and fix that one.
What timeout should I set by default?
For a normal API call, 10–30 seconds is plenty. For AI calls, 60–120 — and if the answer is long, switch to streaming instead. The main rule: the outer link (the browser) should wait longer than the inner one, or the client hangs up before the server can answer.
How is a timeout different from a 504 error?
504 Gateway Timeout is one specific case: a proxy or gateway didn't get an answer from the server behind it. Cloudflare uses its own 524 for the same meaning. Timeouts as a phenomenon are broader — they happen on the client, in the database, and entirely without HTTP.
Should I retry after a timeout?
Carefully. If the request only reads data, retry freely. If it creates something (a payment, an email, an order), remember the server may have finished the work and only failed to answer in time. Retrying without duplicate protection gets your user charged twice.
The AI Architect 2027 course: step by step from computer basics to your own app and first customers. 543 short lessons on your phone.





