Chapter 26 — Critical infrastructure and sector security: operational handoff
Lesson map
Lesson 26 published. Map, example, EX076.
flowchart TD
N0["Learning outcomes and connection"]
N1["26.1 What is handed over"]
N2["26.2 Runbooks that work at 03:00"]
N3["26.3 Health evidence"]
N4["26.4 Running the handover"]
N5["26.6 Practical activity: write and test a runbook"]
N6["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX076"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["26.1 What is handed over"]
N2["26.2 Runbooks that work at 03:00"]
N3["26.3 Health evidence"]
N4["26.4 Running the handover"]
N5["26.6 Practical activity: write and test a runbook"]
N6["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX076"]
[Learning outcomes and connection]
|
v
[26.1 What is handed over]
|
v
[26.2 Runbooks that work at 03:00]
|
v
[26.3 Health evidence]
|
v
[26.4 Running the handover]
|
v
[26.6 Practical activity: write and test a runbook]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX076]
Learning outcomes and connection
After this chapter you should be able to:
- prepare a service for handover from the project team to the people who will run it every day;
- write runbooks that a competent colleague can follow at 03:00 without the author;
- define ownership, service levels and the evidence that the service is healthy;
- run a handover that tests understanding, not just delivers documents.
A service is not finished when it works; it is finished when someone else can run it. Many security controls fail at handover: the person who understood the key rotation leaves the project; the on-call team never learned what Rule C's first action means; the backup restore was rehearsed only by its author.
26.1 What is handed over
A LANTERN handover pack contains:
- Service description — purpose, customers, architecture with zones, conduits and management paths (Chapter 8), and the design properties that must be preserved.
- Ownership — service owner, technical owner, security owner, customer contacts, supplier contacts, with roles and out-of-hours numbers.
- Service levels — what customers can expect (telemetry within 60 s, value age shown, RTO 4 h for the historian) and how they are measured (Chapter 9).
- Runbooks for routine and emergency tasks (see "Runbooks that work at 03:00" below).
- Control cards for every control, with last evidence dates (Chapter 18).
- Monitoring — every rule with its first action, and the self-monitoring checks (Chapter 19).
- Open items — known issues, accepted risks with owners, compensating controls with end dates.
- Evidence record — what has been tested and when (Chapter 29 assembles it).
26.2 Runbooks that work at 03:00
A good runbook is written for a competent colleague who has not seen this service before, under time pressure. It states its purpose and when to use it; prerequisites (access, tools, approvals); steps numbered, each with the exact command or action and what you should see; decision points ("if the relay log shows MAC failures, go to step 9; otherwise step 12"); safety notes where a step could affect operations; rollback; and how to record what was done. Test every runbook by having someone other than its author follow it on a test system and time it.
Core LANTERN runbooks: add or remove a station (standard change); rotate a station key; relay host rebuild; historian restore; respond to each alert rule; suspected relay compromise (first steps and escalation); planned maintenance with Water operations; enabling supplier access.
26.3 Health evidence
The receiving team should be able to answer "is LANTERN healthy?" in two minutes from a dashboard or a daily report: stations reporting / expected; accept, reject and quarantine counts per hour; relay resource headroom (Chapter 1); clock offsets; log-source freshness; last successful backup and last restore test; rules last evaluated; open alerts; silenced rules and their expiry (Chapter 25).
The handover in this chapter found the gap: the health report said "backups OK" but showed no restore-test date. For LANTERN the two-minute check would show 14 stations expected, accept, reject and quarantine counts, clock offsets against the 2 s alert, RLY-01's disk against its 85 % alarm, and any silenced rule with its expiry.
26.4 Running the handover
Handover is a process, not an email. Run it in stages: walk-through of the pack with the receiving team; shadowing, where the receiving team performs routine tasks with the project team watching; reverse shadowing, where the project team watches the receiving team handle an injected test alert and a runbook; and acceptance, where the service owner and receiving team lead sign off with open items listed. Agree an early-life support period in which the project team remains available.
Fallowmere's reverse shadowing used an injected Rule C alert for PS-09.
26.5 Worked example: a handover that found three gaps (synthetic)
During reverse shadowing, the FSTS operations team is given a test Rule C alert for PS-09. The on-call technologist follows the runbook and calls the Water control room — correctly — then checks carrier status as written. The runbook does not mention checking the relay's MAC-failure log, so the technologist calls BayTel; the injected cause was a key mismatch. Second, the key-rotation runbook refers to "the key store" without saying where it is or who has access; the receiving team cannot follow it. Third, the health report shows "backups OK" but not the last restore test date. All three are fixed before acceptance; the handover is delayed by a week, which is far cheaper than discovering the gaps during an incident.
26.6 Practical activity: write and test a runbook
Write the runbook "Respond to Rule C — station silent" using the structure in "Runbooks that work at 03:00" above, including the decision point from the worked example above ("a handover that found three gaps"). Ask a colleague (or, if studying alone, yourself after a day's break) to follow it against the Lab 20 data and note every place they hesitated.
Use the handover's lesson as the decision point: after calling the Water control room, check the relay's MAC-failure log for the silent station before calling BayTel. In the handover test the injected cause was a key mismatch, and the runbook without that step sent the on-call technologist to the carrier. With the Lab 20 data, the same step separates PS-07's lost messages from PS-09's clock offset.
26.7 Common errors and limits
- Documentation delivered without testing that others can use it.
- Runbooks that say "investigate" or "fix the issue".
- Health reports that show configuration success but not effectiveness evidence.
- Handover without shadowing.
- Open items and accepted risks left out to make the service look complete.
- Project team disappearing the day after acceptance.
The PS-09 drill caught the second error: "check carrier status" sent the technologist to BayTel for a key mismatch.
The "backups OK" line without a restore date is the third.
Exercises — chapter 26
EX076 — Runbook critique. Objective: improve a runbook. Inputs: "Key rotation: 1. Generate new key. 2. Put it on the relay and station. 3. Check it works. 4. Delete old key." Constraints: identify at least six problems. Steps: list problems and rewrite. Expected evidence: list and rewrite. Rubric: 3 = missing purpose and prerequisites, no store location or access, no overlap period (Chapter 18), no specific verification, no safety/operations notice, no rollback, deleting old key too early for backlog, no recording; rewrite addresses them; 2 = four or five; 1 = cosmetic; 0 = none.
EX077 — Health report. Objective: define health evidence. Inputs: Chapter 26 ("Health evidence"). Constraints: one screen; ten items maximum. Steps: choose items, give each a green/amber/red threshold, and say who reads it daily. Expected evidence: table. Rubric: 3 = items covering availability (stations reporting), integrity (reject/quarantine), capacity, time, logging, backup/restore, monitoring health and silences; thresholds sensible; reader named; 2 = eight items or thresholds weak; 1 = list without thresholds; 0 = none.
EX078 — Acceptance decision. Objective: decide whether to accept a handover. Inputs: all runbooks tested except historian restore (tested by its author only); two control cards lack evidence dates; one accepted risk (PS09-RTU unsupported) has owner and plan. Constraints: accept, accept with conditions, or reject. Steps: decide and justify, listing any conditions with dates. Expected evidence: paragraph. Rubric: 3 = accept with conditions: independent historian restore test within a short date, evidence for the two controls by a date, early-life support until conditions met; recognizes the accepted risk is properly handled; 2 = decision without conditions; 1 = accept unconditionally; 0 = none.
Chapter synthesis
A service is finished when others can run it. Hand over a pack with description, ownership, service levels, runbooks, control cards, monitoring, open items and evidence. Write runbooks for a competent colleague at 03:00 and test them with someone other than the author. Provide health evidence that shows effectiveness, not only configuration. Run handover as walk-through, shadowing, reverse shadowing and acceptance with early-life support, and fix the gaps it reveals.
Continue studying — chapter 26 of 30
← Chapter 25 · Course chapter index · Chapter 27 →
Continue to the next authored lesson.