Most production incidents start as changes. A solid plan turns a risky rollout into a calm, reversible sequence: agree on risk, set a backout trigger, get the right signatures, and keep people informed. This guide ships an engineering‑focused template you can adopt today, plus a worked example.

When to use a change plan vs a runbook

Runbooks document how to operate or recover a service. A change plan is about the decision to roll forward and the guardrails around that act. Use a change plan for deployments, schema changes, cutovers, or vendor switches that can affect customers. Link to runbooks for the detailed steps.

Decision model: standard, expedited, or emergency

Use three lanes. Standard: routine, low‑risk changes within a window. Expedited: riskier or time‑bound changes with extra comms and sign‑off. Emergency: user harm now; the on‑call and incident commander approve short‑circuit mitigation with immediate follow‑up.

Template fields that make changes safe

  • Summary — one line users would understand.
  • Owner / approver — who runs it and who signs.
  • Risk level — low/medium/high/emergency (see matrix).
  • Blast radius — systems, customers, and data touched.
  • Rollout steps — numbered, reversible actions.
  • Backout plan — specific trigger and steps to undo.
  • Change window — date, time, and freeze conflicts.
  • Communication plan — who to tell, when, and how.
  • Success criteria — telemetry or checks that prove steady state.
  • Evidence — links to PRs, tickets, dashboards.

Risk classification and gates

Classify by impact and likelihood. Gate approvals, comms, and backout triggers by tier. If you cannot name a concrete rollback, the risk is higher than you think.

Risk matrix: impact vs likelihood
Rate changes by impact and likelihood. Gate approvals and comms by tier.

Approvals: CAB‑lite that people actually use

A weekly all‑hands CAB becomes a rubber stamp. Keep approvals tight and accountable: the service owner, an SRE or senior engineer, and support or customer success for high‑risk changes. Capture names before the window opens.

CAB‑lite approvals by risk tier
Increase approval depth as risk rises. Do not send every change to a committee.

Communication playbook

Use short, factual messages. Name impact and next step. Link to status/details.

  • Pre‑announce — “We will upgrade payments on Tue 19:00–19:30 UTC. Brief reads‑only risk. No customer impact expected.”
  • Go‑live — “Change started 19:02 UTC. Monitoring latency and error rate. Next update 15 min or sooner.”
  • Rollback — “Rolled back at trigger (p95 latency > 800ms for 5 min). Investigating cause. No data loss.”
  • Steady state — “Change complete. Metrics normal for 30 min. Closing change.”

Filled example: zero‑downtime DB migration

Summary: Move primary database from v12 to v14 using logical replication; cut over with read‑only window < 60s. Risk: medium (read latency spikes possible). Backout trigger: p95 > 800ms for 5 min or error rate > 1%. Backout: demote new primary, re‑point app to old primary, disable write path until metrics normalize.

  1. Warm up a v14 replica and sync via logical replication.
  2. Health checks green on both clusters; freeze writes for 60s.
  3. Promote v14; point writes to new primary; re‑enable writes.
  4. Monitor latency/error dashboards; lift freeze after 30 min stable.

FAQ

Do we need a CAB? You need accountable approvals. A small, named set of reviewers beats a large meeting.

How strict should the backout trigger be? Strict enough that an on‑call engineer can act without debate. Use objective metrics and a short window.

Where do templates live? In the same repo as the service, reviewed like code, and linked from the PR that ships the change.