Chapter 37 — Critical infrastructure and sector security: operations
Lesson map
Lesson 37 published. Map, example, EX074.
flowchart TD
N0["Learning outcomes and connection"]
N1["37.1 Security operations in OT"]
N2["37.2 Playbooks"]
N3["37.3 Change, access and vulnerability management"]
N4["37.4 Learning without blame"]
N5["37.6 Practical activity"]
N6["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX074"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["37.1 Security operations in OT"]
N2["37.2 Playbooks"]
N3["37.3 Change, access and vulnerability management"]
N4["37.4 Learning without blame"]
N5["37.6 Practical activity"]
N6["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX074"]
[Learning outcomes and connection]
|
v
[37.1 Security operations in OT]
|
v
[37.2 Playbooks]
|
v
[37.3 Change, access and vulnerability management]
|
v
[37.4 Learning without blame]
|
v
[37.6 Practical activity]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX074]
Learning outcomes and connection
After this chapter you can (1) design security operations for OT — monitoring, triage, escalation, coordination with process operations; (2) write operational procedures and playbooks that operators can execute under stress; (3) manage change, access and vulnerabilities in daily operations; and (4) learn from incidents and near misses without blame. It builds on Chapters 18, 29 and 34 and prepares Chapters 38 and 47.
37.1 Security operations in OT
OT security monitoring watches network traffic, logs and process data for signs of misuse. Triage in OT differs from IT: the first question is not "is this malicious?" but "is the process safe?" Escalation paths therefore run to the control room and duty operations manager as well as to security staff. Many utilities do not have a 24/7 security team; the control room is staffed 24/7 and must be able to act on security alarms with clear playbooks, escalating to on-call security specialists.
37.2 Playbooks
A playbook is a short, tested procedure for a specific situation: trigger, immediate safety actions, information to collect, decisions and who makes them, communications, and return to normal. Write for a tired operator at 03:00: numbered steps, clear decision points, phone numbers, no jargon. Test each playbook in exercises and update it after every use. Incident handling guidance such as NIST SP 800-61 Rev. 3 provides general structure; OT playbooks add process safety steps.
LRWA's playbook for an unexpected setpoint change is written to this standard.
37.3 Change, access and vulnerability management
Daily operations keep the system in its designed state: change management (no configuration change without approval and record); access management (joiners, movers, leavers; vendor sessions approved per use); vulnerability management (track advisories such as CISA ICS advisories, assess applicability, plan remediation, Chapter 12). Drift — undocumented changes accumulating — is the commonest way a well-designed system becomes insecure.
At LRWA, the forgotten enterprise reporting script found in the flow review is an example of drift, and change control on every conduit is how TMRP keeps such paths from reappearing.
37.4 Learning without blame
Incidents and near misses are evidence about the system. Reason (2000) contrasted the person approach (blaming individuals) with the system approach (defences, barriers and latent conditions), arguing the latter is more effective. After-action reviews ask what happened, why it made sense to the people involved at the time, which defences worked or failed, and what will change. Blame suppresses reporting, and unreported near misses are lost lessons.
At LRWA, the technician who connected a laptop to the PS3 network in EX074 is treated in this way: the review asks why the jump host felt too slow and what made the shortcut possible.
37.5 Worked example: playbook "unexpected setpoint change" (synthetic)
Trigger: SCADA shows a setpoint change at any station not matching an operator action in the last 5 minutes, or a gateway logs repeated MAC failures. Immediate safety actions: (1) check the station's process values against independent measurements; (2) if outside normal range, restore the previous setpoint locally or switch to local manual via the duty technician; (3) do not shut down remote telemetry. Collect: station, time, old/new values, gateway log extract, operator log. Decide: duty manager decides whether to disable remote writes fleet-wide (gateway configuration switch). Communicate: security on-call within 30 minutes; hospital liaison if PS3/PS4 affected. Return to normal: remote writes re-enabled only after security on-call agrees.
37.6 Practical activity
Run the playbook in a tabletop with a scripted inject sequence (first a single setpoint change, then MAC failures at a second station, then a phone call from the hospital about low pressure). Record decision times and ambiguities and revise the playbook.
Run the injects in order and note the time of each decision. Watch in particular whether the team keeps remote telemetry running, as the playbook requires, and whether the duty manager decides on disabling remote writes across the fleet before the hospital call arrives. Record every point where someone asked what the playbook meant. After the tabletop, rewrite those steps and schedule a second run to check the revisions.
37.7 Common errors and limits
- Security alarms routed to a team that is not staffed when they fire.
- Playbooks written for experts, never tested.
- Change management bypassed "temporarily".
- After-action reviews that end with "operator error".
LRWA's playbook design answers these: security alarms reach the control room, which is staffed every hour, and the security on-call is called within 30 minutes. The playbook is tested in tabletops, and change control has no temporary exceptions. After-action reviews at LRWA look for latent conditions, such as a slow jump host, rather than ending with operator error.
Exercises — chapter 37
EX073 — Playbook review. Objective: improve a playbook. Inputs: §37.5. Steps: identify three weaknesses (e.g., ambiguous thresholds, missing contacts, unsafe default) and rewrite the affected steps. Expected evidence: revised steps. Rubric: 3 = substantive weaknesses fixed with executable steps; 2 = cosmetic changes; 1 = critique only; 0 = none.
EX074 — Near-miss review. Objective: conduct a blameless review. Inputs: a technician connected a laptop to the PS3 control network to download logs, bypassing the jump host; the laptop had not been scanned. No incident followed. Steps: write a one-page review: what happened, why it made sense, defences that failed, system changes. Expected evidence: review. Rubric: 3 = explains local rationality (jump host slow, deadline), identifies latent conditions (no port security, no easy log path), proposes system changes; 2 = some blame; 1 = "retrain technician"; 0 = none.
Chapter synthesis
OT security operations start with process safety and route escalation through the control room. Playbooks must be short, tested and executable under stress. Daily change, access and vulnerability management keep the system as designed, and blameless learning from incidents and near misses improves the defences.
Continue studying — chapter 37 of 50
← Chapter 36 · Course chapter index · Chapter 38 →
Continue to the next authored lesson.