What an incident is — and why you declare one before you find the cause

Here's an odd thing. At Google, the decision "we have an incident" comes down to three questions — and not one of them is "do we know what broke?"
That flips the usual logic. It feels like you should figure it out first and raise the alarm after. It's the other way round: declare first, investigate second. Here's why.
An incident isn't a bug, it's a mode
A bug is a defect in code. It can sit there for months and nobody dies.
An incident is a state where you switch modes: you drop everything else, pull people in, start writing down what's happening. An incident isn't about how big the breakage is — it's about how you respond to it.
So "we have an incident" doesn't mean "all is lost." It means: we're working to a procedure instead of improvising.
The three questions that trigger it
The Google SRE book states the criteria plainly, as a list of questions:
- Do you need to involve a second team to fix the problem?
- Is the outage visible to customers?
- Is the issue still unsolved after an hour's concentrated analysis?
One "yes" is enough. Declare it.
Look closely at the third one. An hour of digging with no result is grounds by itself. Not "it got worse" — just "an hour passed and I still don't understand." That's exactly where most people stall: it always feels like fifteen more minutes will crack it.
The book puts it even more bluntly: it is better to declare an incident early, find a simple fix and close it out, than to spin up the whole framework hours into a growing problem.
The commander doesn't fix anything
The least obvious part of Google's scheme is splitting the roles. There are several, but three matter at the start:
- Incident commander. Holds the whole picture: what's known, who's doing what, what's next. Doesn't touch the code.
- Ops lead. The only person touching the system: rolling back, restarting, changing config.
- Communications lead. Writes updates — to the team and to whoever is waiting outside.
Why doesn't the commander fix things? Because ten minutes into the logs, a person drops out of reality. They stare at one spot and don't notice customers haven't been able to pay for an hour.
This isn't bureaucracy. It's protection against six people editing the same file at once while nobody explains what's going on.
What to do when you're alone
You have no second team and no communications lead. It still works — the roles just become modes you switch between:
- Stop the bleeding, then look for the cause. The first move in an incident is a rollback to the last working version. Not "let me just patch it quickly" — actually restore what worked.
- Open a text file and write timestamps. "14:20 saw 500s. 14:24 rolled back. 14:31 responses recovered." In two hours you won't remember the order of events, and the order is the valuable part.
- Say out loud who's waiting. Even if it's three people in a chat — write "I see it, I'm on it." Silence is worse than bad news.
- Set yourself a one-hour timer. Didn't find it? Change the approach: pull someone in, roll back further, turn the feature off entirely.
Later, once it's quiet, you sit down and work through it: what happened, and why it took that long. That's a postmortem — a separate document written without the rush.
And so you don't have to recall commands from memory at three in the morning, the steps from point 1 belong in a runbook, written in advance.
When an incident counts as closed
Nearly everyone gets this wrong: the service responds, so we're done here. No.
Restoring service and closing the incident are two different events. Three things belong in between:
- Make sure it isn't a coincidence. Wait until the numbers hold steady for at least a few minutes. A restart often "fixes" things for five minutes because the problem simply builds up again.
- Check the damage. While everything was down, things got lost: unsent emails, orders stuck halfway, background jobs that never ran. Catching those up by hand is part of the incident, not a task for later.
- Tell the people who were waiting that it's over. Silence after "I'm on it" reads as "still down."
Only now can you drop the mode. Closing usually sounds like this: "service is healthy, lost jobs caught up, review tomorrow." With a date and an owner on that review — otherwise it never happens.
Is an incident the same as a bug?
No. A bug is a defect in code; it goes on the task list. An incident is a mode of responding to something happening right now: roles, a clock running, priority over everything else. One bug can cause an incident — or sit in the backlog for six months.
What if it fixed itself?
It was still an incident. "Fixed itself" usually means load dropped or something restarted — the cause is still there and will be back. Those are the most valuable ones to review: a failure with no damage is a free lesson.
Should I declare incidents with ten users?
Yes — for the habit. The point isn't audience size, it's having a rehearsed order of moves: stop, record, tell people, review. At ten users that habit is cheap to build. At ten thousand it's expensive.
The AI Architect 2027 course: step by step from computer basics to your own app and first customers. 543 short lessons on your phone.





