Chapter 39 — Maintainability and change control for playbooks
Lesson map
Lesson 39 published. Map, example, EX077.
flowchart TD
N0["Learning outcomes and connection"]
N1["39.1 Why playbooks decay"]
N2["39.2 Playbooks as code"]
N3["39.3 Triggers and regression tests"]
N4["39.5 Deprecation"]
N5["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX077"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["39.1 Why playbooks decay"]
N2["39.2 Playbooks as code"]
N3["39.3 Triggers and regression tests"]
N4["39.5 Deprecation"]
N5["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX077"]
[Learning outcomes and connection]
|
v
[39.1 Why playbooks decay]
|
v
[39.2 Playbooks as code]
|
v
[39.3 Triggers and regression tests]
|
v
[39.5 Deprecation]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX077]
Learning outcomes and connection
After this chapter you can (1) treat response playbooks as versioned, owned, tested artefacts; (2) define triggers that force review; (3) write regression tests for playbook steps against a lab environment; and (4) deprecate outdated procedures safely. It builds on Chapters 14, 31 and 37 and prepares Chapters 40, 46 and 48.
39.1 Why playbooks decay
Playbooks describe actions in tools that change monthly: cloud commands are renamed, permission models change, approval chains shift with reorganisations, phone numbers go stale. A playbook last reviewed two years ago is a hazard: responders follow it under pressure and discover mid-incident that step 4 fails or step 6 calls someone who left. Maintainability — the ease with which a system can be kept correct as its environment changes — is a dependability attribute (Avižienis et al., 2004) and applies to procedures as much as to code.
39.2 Playbooks as code
Store playbooks in version control with: a unique ID and semantic version; an accountable owner (a named role and person); dependencies (tools, APIs, permissions, contacts, other playbooks); review date; triggers for unscheduled review; and machine-checkable steps where possible (a CACAO-style structure, Chapter 31, or simple YAML). Changes go through review like code, with a changelog. Human-only steps remain text, but their dependencies are declared so that checks can test them. PB-CLOUD-07 in the worked example shows what the missing fields cost: v1.3 declared no CLI version and no approving role.
39.3 Triggers and regression tests
Review triggers: a dependency changes (tool upgrade, API deprecation notice, permission model change); an organisational change affects any named role; the playbook was used in an incident or exercise (lessons learned); the review date passes. Regression tests run in a lab tenant: each automated step executes against synthetic targets and checks its expected effect; contact steps check that each named role resolves to a current person with a working number (a directory lookup, not a call); permission steps check that the integration identity still holds the declared scopes.
39.4 Worked example (synthetic): an outdated cloud playbook at HVSS
Playbook PB-CLOUD-07 "Contain compromised cloud access key" v1.3, last reviewed 20 months ago. Review findings: (1) step 3 uses a command-line flag removed in the current CLI version — the regression test fails; (2) step 5 requires approval from the "Infrastructure Manager", a role abolished in a reorganisation — now split between "Cloud Platform Lead" and "Service Owner"; (3) step 7 says "delete the key", which destroys evidence of key usage linking; current guidance is deactivate, preserve, then delete after investigation; (4) no step for checking other keys created by the same principal. Revised v2.0: updated command with a version-pinned test; approval by Cloud Platform Lead with Service Owner informed; deactivate-not-delete; added enumeration step; triggers declared (CLI major version, IAM model changes, org chart changes); quarterly review date; regression test in the lab tenant run weekly by the pipeline. v1.3 marked deprecated with a pointer to v2.0 and removed from the responders' quick index; kept in history for audit of past cases that used it.
39.5 Deprecation
Never delete a playbook used in past cases: auditors and investigators need to know what responders were told to do at the time. Mark deprecated, point to the replacement, remove from active indexes and record the date. Cases record the playbook version followed.
PB-CLOUD-07 v1.3 follows this rule in the worked example from the previous subsection. It is marked deprecated, points to v2.0 and is gone from the responders' quick index, but it remains in history. A reviewer of a past cloud-key case can then see that responders were told to delete the key and judge their actions against that instruction.
39.6 Errors and limits
- Playbooks without owners or review dates.
- Contacts maintained manually in text.
- Testing only after an incident exposes failure.
- Deleting old versions needed for audit.
PB-CLOUD-07 v1.3 shows the first three errors in this list at once. It had gone 20 months without review. Its approval step still named the abolished Infrastructure Manager because contacts lived in text rather than a directory lookup. Its broken CLI flag would have been found mid-incident if the review had not run the regression test. The v2.0 fix is a weekly pipeline test in the lab tenant, with v1.3 kept for audit.
Exercises — chapter 39
EX077 — Metadata. Objective: define playbook metadata. Inputs: PB-CLOUD-07 v2.0. Steps: write its metadata block (YAML-like) with ID, version, owner, dependencies, triggers, review date, tests and deprecation link. Expected evidence: block. Rubric: 3 = complete and specific; 2 = one gap; 1 = minimal; 0 = none.
EX078 — Review schedule. Objective: plan maintenance. Inputs: 40 playbooks of varying criticality; two engineers available for 4 hours per week each for playbook work; full review takes 3 hours, quick check 30 minutes. Steps: design a schedule meeting quarterly full review for 10 critical playbooks and semi-annual for the rest, with monthly quick checks of critical ones; show it fits. Expected evidence: calculation. Rubric: 3 = correct load calculation (≈ 6.9 h/week of 8 available, so it fits with about 1 h/week of margin, which should be reserved for trigger-driven reviews); 2 = partial; 1 = vague; 0 = none.
Chapter synthesis
Playbooks decay with their environment. Version them, give them owners, declare dependencies and triggers, test automated and contact steps continuously and deprecate rather than delete, so that procedures stay correct and past decisions stay auditable.
Continue studying — chapter 39 of 50
← Chapter 38 · Course chapter index · Chapter 40 →
Continue to the next authored lesson.