MTTR (mean time to recovery)
MTTR (mean time to recovery, or to repair) is the average time from the start of a failure until the service works again.
The R is not used consistently: recovery, repair, restore, resolve or respond. Recovery (service works again) and resolve (the cause is fixed) can be hours apart, so say which one a number means.
Recovery time breaks into detection, acknowledgement and repair. Frequent checks and checks from several regions shorten detection; alerts that reach the right person shorten acknowledgement; rollbacks and runbooks shorten repair.
At a glance
MTTR = total downtime ÷ number of incidents 3 incidents, 90 min in total: MTTR = 30 min
In OpenUptime
OpenUptime incidents record when a monitor went down, when someone acknowledged the incident and when it recovered. Reminders and escalation repeat the alert while nobody has acknowledged it.
Questions
What is a good MTTR?
It depends on the target: at 99.9% a month you have about 43 minutes in total, so recovery needs to be well under that. At 99.99% it has to be automatic.
Does MTTR include detection time?
For recovery, yes: it starts when the failure starts, not when someone noticed. That is why faster detection lowers MTTR.
Related terms
Monitor it with OpenUptime
Checks from one or several regions, alerts to the tools you use, and a status page for your customers. Free to start, open source under the MIT license.
MTTR(平均恢复时间)
MTTR(平均恢复时间,也作平均修复时间)是从故障开始到服务重新正常工作的平均时长。
R 的含义并不统一:recovery(恢复)、repair(修理)、restore(复原)、resolve(解决)或 respond(响应)。恢复(服务重新可用)和解决(根因修复)之间可能相差几个小时,所以要说明数字指的是哪一种。
恢复时间可以拆成发现、确认和修复三段。更频繁的检查和多区域检查能缩短发现时间;把告警送到对的人能缩短确认时间;回滚和处理手册能缩短修复时间。
速览
MTTR = 总停机时间 ÷ 事件数 3 次事件共 90 分钟:MTTR = 30 分钟
在 OpenUptime 中
OpenUptime 的事件会记录监控何时宕机、何时有人确认、何时恢复。在无人确认时,提醒和升级机制会重复发送告警。
常见问题
MTTR 多少算好?
取决于目标:按月 99.9% 总共只有约 43 分钟,恢复时间必须远低于这个值。到了 99.99%,恢复必须自动完成。
MTTR 包括发现故障的时间吗?
按“恢复”的定义是包括的:从故障开始算起,而不是从有人发现开始。这也是缩短发现时间能降低 MTTR 的原因。