Post-mortem
The incident doc that blames a person
The outage of 14 March
- Shape
- When it breaks
- Plates
- lede · timeline · worked · numbered
- Why
- A post-mortem is read by people who were not there, so the clock has to be legible: `timeline` and not a bulleted list, because the gaps are the story — eleven minutes to notice and ninety to understand is the finding. `worked` on the failing request rather than the healthy one. End with `numbered` changes that already happened, or this is a document about feelings.
For one hour and fifty-one minutes, about a third of requests failed. Nobody did anything wrong. A limit set in 2021 met a traffic pattern from 2026.
The hour and fifty-one minutes
09:14
Deploy goes out
Unremarkable, and not the cause
09:31
First failures
Below the alert threshold, so nothing fires
09:42
A customer tells us
Eleven minutes of failures nobody saw
11:12
Cause found
The connection pool, capped at fifty since 2021
11:16
Resolved
Cap raised. The fix took four minutes
What one failing request did
in
waits
out
- 1.
The alert now fires on error rate, not error count. Shipped 15 March. This is what cost us the eleven minutes.
- 2.
The pool size is derived, not written down. Shipped 18 March. It scales with the instance rather than with somebody's memory of 2021.
- 3.
Connection saturation is on the dashboard. It was measurable all along. Nobody was looking at it.
This is a sample. To author the same thing about your own subject, just name it. Your assistant can read this template, and will write you the same document about what you know.
Using the Ario tools, make me the "Post-mortem"
template for the incident on the 3rd.
Show me the shape before you build anything.New here? How to connect one.
Other samples