What is a postmortem?
A postmortem, also called an incident review or retrospective, is a written analysis after an incident: the timeline, the impact on users, the causes and contributing factors, what went well and badly in the response, and concrete action items with owners. A blameless postmortem focuses on how the system allowed the failure, not on who made a mistake.
Why a postmortem matters
Incidents are expensive lessons. Without a postmortem the lesson stays in a few people's heads and the same failure returns months later.
Blameless writing matters because people hold back details when they expect blame, and the detail held back is often the cause. Google's Site Reliability Engineering book describes the practice at length, and many teams publish theirs.
How to measure a postmortem
impact (failed requests) ≈ error rate × request rate × duration timeline, in one timezone: start, detection, acknowledgement, mitigation, resolution
Worked example
During an incident 18% of requests failed for 34 minutes. Normal traffic at that hour is 1,200 requests per minute.
- Requests in the window = 1,200 × 34 = 40,800.
- Failed requests ≈ 0.18 × 40,800 = 7,344.
- Stating the impact as about 7,344 failed requests, and how many users or orders that represents, is more useful than "partial outage".
Result: A number like this also tells you how much of the error budget the incident used, which decides how urgent the action items are.
Common mistakes
- A person as the root cause. "An engineer ran the wrong command" is a symptom. Ask why the command was possible and why nothing caught it.
- Action items without owners or dates. They do not get done.
- Writing it weeks later. Details and logs fade. Draft it within a few days.
- Only one root cause. Most incidents have several contributing factors; list them all.
How Vigil handles a postmortem
- Every Vigil incident has a postmortem next to its timeline, updates and failure evidence.
- "Analyse with AI" drafts a summary, likely root cause, fix steps and a postmortem from the evidence, which you edit before anyone reads it. AI never mutes or silences alerts.
- Time Detective lists what happened around the start time (failed checks, alerts, host peaks, errors, log errors, deploys and traces), which is most of the timeline already written.
Postmortem: frequently asked questions
What should a postmortem include?
A summary, the impact, a timeline, causes and contributing factors, what went well and what did not, and action items with owners and dates.
What does blameless mean?
That the review assumes people acted reasonably with what they knew at the time, and looks for the conditions that allowed the mistake, so people can share details without fear.
Who should write it?
Usually the incident owner or someone closely involved, with input from everyone who responded. A review meeting then checks the facts and agrees the actions.