Chapter 18 — Containment and recovery control implementation

Chapter 18 — Containment and recovery control implementation

Lesson map

Lesson 18 published. Map, example, EX052.

flowchart TD
  N0["Learning outcomes and connection"]
  N1["18.1 Containment actions as products"]
  N2["18.2 Recovery"]
  N3["18.3 Verification"]
  N4["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX052"]

Logical drawing

flowchart LR
  N0["Learning outcomes and connection"]
  N1["18.1 Containment actions as products"]
  N2["18.2 Recovery"]
  N3["18.3 Verification"]
  N4["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX052"]

[Learning outcomes and connection]
    |
    v
[18.1 Containment actions as products]
    |
    v
[18.2 Recovery]
    |
    v
[18.3 Verification]
    |
    v
[Chapter synthesis]
    |
    v
[Worked example] --> [Exercise EX052]

Learning outcomes and connection

You will be able to (1) implement pre-built containment actions with tested side effects and rollback; (2) implement and time recovery procedures, including restoring from backup; and (3) verify that containment and recovery worked. Chapters 6–8 showed containment ideas; this chapter turns them into implemented controls the duty analyst can trigger safely.

18.1 Containment actions as products

Each common containment action becomes a documented, tested capability: disable account and revoke sessions; isolate an endpoint through the agent; move a VM to quarantine; block an address or domain at the edge and DNS resolver; disable a public feature behind a flag; rotate a service credential. For each: trigger (who may run it), procedure or automation, expected side effects, verification step and rollback. Test each quarterly in a non-production environment and record the time taken. In the permit-app-2 case, moving the VM to quarantine was one of these pre-built actions.

18.2 Recovery

NIST SP 800-184 describes recovery planning: prioritized restoration, validated clean sources, and improvement. Recovery controls include immutable or offline backups, documented restore procedures with measured times, clean build images, and validation steps before reconnecting to production. Restore tests should restore to an isolated environment and check data integrity, not just that the job "succeeded". Fernmoor's timed restore of appt-db-1 shows why measurement matters: the first run took far longer than planned because the restore tool was single-threaded, and point-in-time recovery was not possible at all. Neither problem would have appeared in a backup job's success report. Both were fixed and retested before the plan was relied upon.

18.3 Verification

Containment is complete only when verified: the account cannot sign in (test), sessions are gone (check active sessions list), the host cannot reach anything but the evidence share (connectivity test), the blocked address produces denials (edge log). Recovery is complete when the service owner accepts functional tests and monitoring is in place for re-infection. In the reception PC case, the action had been recorded as complete when the isolation was only queued, which shows why the connectivity check matters. Fernmoor's runbooks now include a verification step for every containment action, with the evidence of verification recorded in the case.

18.4 Worked example (synthetic): timed restore of appt-db-1

Fictional. Chapter 1 noted the appointment database restore had never been timed. The team ran a test:

Results: recovery time 107 minutes on first run, 64 on retest, against a tier-2 objective of 4 hours — met. Recovery point: up to 24 hours of appointments could be lost because WAL archiving was off — an objective of 1 hour was not met. A change enabled WAL archiving to the separate backup account; a retest showed point-in-time recovery working.

18.5 Common errors and limits

• Containment that has never been tested and fails at 03:00.
• "Backup succeeded" without a restore test.
• Restoring into production before validating that the backup is clean.
• Forgetting to verify containment; the attacker's second account still works.

Fernmoor's restore test shows the second and third errors avoided: the database was restored to an isolated host, checked against row counts and checksums, and only then smoke-tested. The quarterly action tests address the first, and the verification step in each containment runbook addresses the fourth. The capstone's DNS sinkhole test, found 120 days old, shows how easily such tests slip.

Exercises — chapter 18

Attempt each exercise before opening the worked results. Every case, organization, person, address and number in an exercise is synthetic unless a cited public source is named.

EX052 — Action card. Steps: write the action card for "move VM to quarantine segment". Rubric: 3 = trigger/authority, steps, side effects, verification, rollback, test record; 2 = missing two; 1 = steps only; 0 = none.

EX053 — Restore test plan. Steps: write a restore test plan for billing-db. Rubric: 3 = isolated target, timed steps, integrity checks, RPO/RTO comparison, findings and owner; 2 = no integrity checks; 1 = vague; 0 = none.

EX054 — Verify containment. Steps: list verification checks for containing a compromised svc-reports cloud key. Rubric: 3 = key disabled/deleted, API call with key fails, no other keys for principal, audit shows no new keys, resources created by key reviewed; 2 = three; 1 = one; 0 = none.

Chapter synthesis

Turn containment and recovery into tested products with authority, side effects, verification and rollback; time restores in isolation and compare with recovery objectives; and never call an action done until verification proves it.

Continue studying — chapter 18 of 30

← Chapter 17 · Course chapter index · Chapter 19 →

Continue to the next authored lesson.