OpenUptime

Postmortem template for outages

A structure for writing up an outage after it is over: what happened, who it affected, why, and what will change so it does not happen again.

Templates

Replace the parts in [brackets].

Full postmortem

For your team, after any outage customers noticed.

# [Incident title], [date]

## Summary
[Two or three sentences: what broke, for how long, and how it was fixed.]

## Impact
- Duration: [start] to [end] UTC ([duration])
- Affected: [features, regions, share of customers or requests]
- Data: [lost / delayed / none]

## Timeline (UTC)
- [hh:mm] [First failed check / first alert]
- [hh:mm] [Someone acknowledged it]
- [hh:mm] [Cause found]
- [hh:mm] [Fix deployed]
- [hh:mm] [Back to normal]

## Root cause
[What failed and why. Separate the trigger (what started it) from the cause (why it could).]

## Resolution
[What you did to restore service.]

## What went well
- [e.g. the alert fired within two minutes]

## What went wrong
- [e.g. the runbook pointed to an old dashboard]

## Action items
| Action | Owner | Due |
| --- | --- | --- |
| [Change] | [Name] | [Date] |

Public summary

For your status page or customers, once the incident is resolved.

On [date], from [start] to [end] UTC, [what customers experienced, e.g. most API requests failed].

What happened: [the cause, in one or two plain sentences].

What we're changing:
- [Change one]
- [Change two]

We're sorry for the disruption. If you were affected and need help, contact [support address].

Review meeting questions

To run the review meeting itself.

1. When did the problem start, and when did we first know? What closed the gap, or would have?
2. Did the alert reach the right person, and was it clear what was wrong?
3. What slowed us down: access, missing logs, unclear ownership?
4. Could a customer have told us sooner than our monitoring did?
5. Which one change would have prevented this, and which would have made it shorter?

Writing tips

  • Keep it blameless: describe what the system and the process allowed, not who made the mistake. People stay honest when they are not on trial.
  • Write it while it is fresh, usually within a few days, and build the timeline from check results and alert times rather than memory.
  • Every action item needs an owner and a date, or it will not happen.
  • Keep the public version short and free of internal names, hostnames and detail that would help an attacker.

In OpenUptime

  • Each incident keeps its updates in order with their times, a starting point for the timeline.
  • An incident has a short Root cause field and a longer Postmortem write-up.
  • Both stay private until the incident is resolved and you choose to publish them on the status page.

Publish these on a status page

Status pages, incident updates and subscriber emails are included on the Free plan. No card needed.

Questions

What is a blameless postmortem?

A review that looks for the conditions that let a failure happen, not for someone to blame. It assumes people acted reasonably with what they knew, which makes it easier to learn what really went wrong.

When should we write a postmortem?

For any outage customers noticed, any data loss, and any near miss that could have been serious. Small incidents can share one short review.

Should the postmortem be public?

A short public summary builds trust after a visible outage. Keep the full internal document for the team; publish what customers need: impact, cause in plain words, and what is changing.

故障复盘模板

故障结束后用来整理复盘的结构:发生了什么、影响了谁、为什么发生,以及之后要改什么,避免再次发生。

模板

替换 [方括号] 中的内容。

完整复盘

写给团队,用于任何客户感知到的故障。

# [故障标题],[日期]

## 摘要
[两三句话:什么坏了、持续多久、如何修复。]

## 影响
- 持续时间:北京时间 [开始] 至 [结束](共 [时长])
- 影响范围:[功能、地区、受影响的用户或请求比例]
- 数据:[丢失 / 延迟 / 无影响]

## 时间线(北京时间)
- [hh:mm] [第一次检查失败 / 第一条告警]
- [hh:mm] [有人确认告警]
- [hh:mm] [找到原因]
- [hh:mm] [修复上线]
- [hh:mm] [恢复正常]

## 根因
[什么失效了、为什么。把触发因素(是什么引发的)和根本原因(为什么会被引发)分开写。]

## 处理过程
[恢复服务时做了什么。]

## 做得好的地方
- [例如:告警在两分钟内发出]

## 需要改进的地方
- [例如:处理手册指向了旧的监控面板]

## 改进项
| 改进项 | 负责人 | 截止日期 |
| --- | --- | --- |
| [改动] | [姓名] | [日期] |

公开版摘要

故障恢复后,发布到状态页或发给客户。

[日期] 北京时间 [开始] 至 [结束],[用户遇到的情况,例如大部分 API 请求失败]。

发生了什么:[用一两句通俗的话说明原因]。

我们的改进:
- [改进一]
- [改进二]

给你带来的不便,我们深表歉意。如果你受到影响并需要帮助,请联系 [支持邮箱]。

复盘会议问题

用于主持复盘会议。

1. 问题从什么时候开始?我们最早什么时候知道?是什么缩短了这段时间,或者本可以缩短它?
2. 告警有没有送到对的人手里?看告警能不能明白哪里出了问题?
3. 是什么拖慢了处理:权限、缺少日志,还是职责不清?
4. 客户会不会比我们的监控更早发现问题?
5. 哪一项改动能避免这次故障?哪一项能让它持续更短?

写作建议

  • 坚持无责复盘:描述系统和流程允许了什么,而不是谁犯了错。人只有在不被追责时才会如实陈述。
  • 趁记忆还新的时候写,通常在几天之内;时间线依据检查结果和告警时间来整理,不要凭记忆。
  • 每个改进项都要有负责人和截止日期,否则不会落地。
  • 公开版保持简短,不要出现内部名称、主机名或可能帮助攻击者的细节。

在 OpenUptime 中

  • 每个故障都按时间顺序保存所有更新,可以直接作为时间线的起点。
  • 每个故障都有简短的“根因”字段和较长的“复盘”正文。
  • 这两部分在故障恢复且你选择发布之前都不会公开,发布后显示在状态页上。

把它们发布到状态页上

免费版即可使用状态页、故障通报和订阅邮件,无需绑定银行卡。

常见问题

什么是无责复盘?

一种寻找“是什么条件让故障发生”的复盘,而不是寻找责任人。它假设每个人都根据当时掌握的信息做出了合理判断,这样更容易弄清问题到底出在哪里。

什么时候需要写复盘?

任何客户感知到的故障、任何数据丢失,以及任何差点酿成严重后果的事件都应该写。小问题可以合并成一份简短的复盘。

复盘需要公开吗?

在明显的故障之后,一份简短的公开摘要有助于重建信任。完整的内部文档留给团队,对外只发布用户需要的内容:影响、通俗的原因说明,以及后续改进。