Chapter 8 — Critical infrastructure and sector security: difficult cases
Lesson map
Lesson 08 published. Map, example, EX036.
flowchart TD
N0["Learning outcomes and connection"]
N1["8.1 What makes a case difficult"]
N2["8.2 Cascades and backup clocks"]
N3["8.3 Restoration priorities need explicit weights"]
N4["8.4 When a trusted supplier is the cause"]
N5["8.5 Extortion"]
N6["8.6 Ambiguous insiders"]
N7["8.7 Conflicting obligations"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX036"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["8.1 What makes a case difficult"]
N2["8.2 Cascades and backup clocks"]
N3["8.3 Restoration priorities need explicit weights"]
N4["8.4 When a trusted supplier is the cause"]
N5["8.5 Extortion"]
N6["8.6 Ambiguous insiders"]
N7["8.7 Conflicting obligations"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX036"]
[Learning outcomes and connection]
|
v
[8.1 What makes a case difficult]
|
v
[8.2 Cascades and backup clocks]
|
v
[8.3 Restoration priorities need explicit weights]
|
v
[8.4 When a trusted supplier is the cause]
|
v
[8.5 Extortion]
|
v
[8.6 Ambiguous insiders]
|
v
[8.7 Conflicting obligations]
|
v
[Worked example] --> [Exercise EX036]
Learning outcomes and connection
After this chapter you should be able to:
- Recognize the features that make a case difficult: interdependence, time pressure, conflicting duties, incomplete evidence and harm in every option.
- Analyse cascading failures across sectors and set restoration priorities with explicit weights.
- Structure decisions about trusted-supplier failures, extortion and ambiguous insider events.
- Handle conflicting obligations to regulators, customers, partners and law enforcement without deceiving any of them.
- Communicate under uncertainty in ways that later evidence will not discredit.
This chapter brings together the decision rules of Chapter 1, the evidence discipline of Chapter 3, the ethics of Chapter 4 and the integration of Chapter 7, and applies them where they strain. Chapter 9 then asks how a leader prepares an organization for such cases.
8.1 What makes a case difficult
Easy cases have a right answer that is hard to execute. Difficult cases have no option without harm, and the specialist must help choose which harm, quickly, with incomplete evidence, under scrutiny. Five features recur: interdependence, so that a decision for one service changes outcomes in others; time pressure, because backup supplies, storage and patience run down; conflicting duties to different stakeholders; ambiguous evidence, especially about cause and intent; and irreversibility, because some actions — paying, disconnecting, announcing — cannot be undone. The disciplines of earlier chapters still apply; what changes is that they must be prepared in advance, because there will be no time to invent them.
8.2 Cascades and backup clocks
Infrastructures depend on one another physically (power for pumps), digitally (telemetry over telecoms), geographically (shared ducts and sites) and logically (markets, staffing) (Rinaldi, Peerenboom and Kelly, 2001). Network studies show that interdependent systems can fail more abruptly than either alone (Buldyrev et al., 2010). For a specialist the practical tool is the backup clock: for each essential site, how long its own backup (generator fuel, storage, battery, manual workaround) lasts after a dependency fails. Harm begins when a clock runs out. Restoration decisions — which feeder to re-energize, which site to visit first — are choices about which clocks to stop.
The December 2015 intrusions at three Ukrainian electricity distribution companies, described in an industrial control systems alert, caused outages affecting approximately 225,000 customers; after power was restored, the companies continued to run under constrained operations (CISA, IR-ALERT-H-16-056-01). The case is a reminder that the end of an outage is not the end of a cyber-induced incident: control systems may not be trusted for some time, and restoration and continued operation may depend on manual work.
8.3 Restoration priorities need explicit weights
Restoration order is usually set by habit: largest load first, or quickest job first. Neither is designed to minimize harm. Ordering jobs to minimize total weighted completion time is a classic scheduling result (Smith, 1956); once backup clocks act as due dates, the problem becomes one of weighted tardiness, which is much harder in general — but for a handful of feeders you can simply enumerate every order. The harder part is the weights. Is an hour without power in a hospital worth the same as an hour without power in a home? Treating every person-hour equally is itself a value judgement, and usually a poor one when some sites carry life-critical functions. Weights must be agreed in advance with emergency management and recorded; the analysis then shows the order those weights imply and how sensitive it is to a backup failing.
8.4 When a trusted supplier is the cause
Supplier failures are difficult because the organization's own processes trust the supplier. The July 2024 faulty-update outage required affected organizations to recover many endpoints by hand (Microsoft, 2024). The decisions in such a case include: whether to disconnect the supplier's channel before knowing whether the cause is error or compromise; which systems to recover first (§8.3's restoration logic applies to endpoints as well as feeders); whether to delay updates in future and accept the security cost of slower protection; and how to keep detection while the detection tool itself is the problem. Preparing for this case means staged update rings for critical systems, independent recovery media, and a minimum capability that does not depend on the supplier.
8.5 Extortion
Ransomware and data-extortion cases force a decision about payment while services are degraded. A specialist does not decide whether to pay — that belongs to the executive, advised by counsel — but structures the decision: the realistic recovery time with and without decryption (decryptors are often slow and imperfect); the service and safety consequences of each timeline; the legal position, including sanctions risks, which must be checked with counsel; insurer conditions; the ethical and strategic consequences of funding criminal activity; and the evidence that would change the assessment. In May 2021 the Colonial Pipeline Company proactively shut down its pipeline system in response to a ransomware attack and restarted it six days later (U.S. Department of Energy; see also CISA, 2023), illustrating that the decision to stop an essential service can itself be a precaution taken under uncertainty, with large consequences for others. Recovery readiness — tested backups, manual operation, clean rebuild capability — is what gives an organization freedom in this decision; it must exist before the case arises.
8.6 Ambiguous insiders
The Hollin event (Chapter 3) could be misuse of a shared vendor account, a vendor engineer's unrecorded legitimate change, or something else. Insider cases are difficult because the people involved are colleagues or partners, evidence is ambiguous, and premature accusation is harmful and sometimes unlawful. The water-sector case discussed in Chapters 2 and 3, whose first account was later disputed (WUSF, 2023), shows how public narratives can run ahead of evidence. Practice: act on the access, not the person (suspend the shared account, require named accounts), preserve evidence, involve HR and legal early, keep accusations out of communications, and remember that human error is usually a symptom of system design (Reason, 2000).
8.7 Conflicting obligations
In a significant incident, a specialist may face, at once: a regulator's reporting deadline; law enforcement asking that some details remain confidential; a partner operator asking whether it is exposed; customers needing safety information; a supplier asking for delay before anything is said about its product; and insurers' conditions. The resolution principles are: never deceive anyone; separate what you say from what you withhold, and withhold only with a reason you could defend; give safety information first; meet legal duties, checking with counsel when duties conflict; and record each decision and its reason. Coordinated vulnerability disclosure (Chapter 4) is one structured way to manage the supplier conflict.
8.8 Worked example: restoration after a regional outage (synthetic)
A cyber incident at Merrow Energy's control centre has de-energized four feeders. Operators can safely restore one feeder at a time, manually. Each feeder serves essential sites with backup clocks: Hollin water works has six hours of storage; Fenley pumping four hours; the Harbour hospital has 48 hours of generator fuel; the container terminal two hours of battery; a fuel depot eight hours. Homes have no backup. The regional forum's emergency cell must recommend an order. Lab 8 enumerates every order, first with every person-hour weighted equally.
8.9 Practical activity: restoration order
# Lab 8 - restoration order after a regional outage, with backup clocks running (offline, stdlib only)
# Synthetic: after a cyber-induced distribution outage, Merrow Energy can re-energize one feeder at a time.
# Each feeder carries essential sites whose backup lasts a fixed number of hours; after that, people lose service.
from itertools import permutations
FEEDERS = { # feeder: (hours to restore safely, [(site, people served, backup hours)])
"F1 Harbour": (4, [("homes", 40000, 0), ("Harbour hospital", 3000, 48)]),
"F2 Hollin": (3, [("Hollin water works", 70000, 6)]),
"F3 Fenley": (2, [("homes", 18000, 0), ("Fenley pumping", 25000, 4)]),
"F4 Port": (5, [("container terminal", 2000, 2), ("fuel depot", 15000, 8)]),
}
def harm(order):
t, total = 0, 0
for f in order:
t += FEEDERS[f][0]
for site, people, backup in FEEDERS[f][1]:
total += people * max(0, t - backup)
return total
res = sorted((harm(o), o) for o in permutations(FEEDERS))
print("best five orders (person-hours without service):")
for h, o in res[:5]: print(f" {h:9,d} {' > '.join(o)}")
print(f"worst: {res[-1][0]:,d} {' > '.join(res[-1][1])}")
largest = sorted(FEEDERS, key=lambda f: -sum(p for _, p, _ in FEEDERS[f][1]))
print(f"'largest load first' {' > '.join(largest)}: {harm(largest):,d}")
shortest = sorted(FEEDERS, key=lambda f: FEEDERS[f][0])
print(f"'quickest job first' {' > '.join(shortest)}: {harm(shortest):,d}")
# What if the hospital generator fails at hour 2 (backup 48 -> 2)? Re-solve.
FEEDERS["F1 Harbour"][1][1] = ("Harbour hospital", 3000, 2)
res2 = sorted((harm(o), o) for o in permutations(FEEDERS))
print(f"after hospital generator failure, best: {res2[0][0]:,d} {' > '.join(res2[0][1])}")
Expected output:
best five orders (person-hours without service):
510,000 F3 Fenley > F2 Hollin > F1 Harbour > F4 Port
589,000 F2 Hollin > F3 Fenley > F1 Harbour > F4 Port
600,000 F3 Fenley > F1 Harbour > F2 Hollin > F4 Port
631,000 F1 Harbour > F2 Hollin > F3 Fenley > F4 Port
642,000 F1 Harbour > F3 Fenley > F2 Hollin > F4 Port
worst: 1,299,000 F4 Port > F1 Harbour > F3 Fenley > F2 Hollin
'largest load first' F2 Hollin > F1 Harbour > F3 Fenley > F4 Port: 681,000
'quickest job first' F3 Fenley > F2 Hollin > F1 Harbour > F4 Port: 510,000
after hospital generator failure, best: 531,000 F3 Fenley > F2 Hollin > F1 Harbour > F4 Port
Interpretation. With equal weights, the best order — Fenley, Hollin, Harbour, Port — loses 510,000 person-hours of service; the worst loses 1,299,000. The habit "largest load first" costs 681,000, a third more than the best; "quickest job first" happens to coincide with the best here, but only because short jobs and short backup clocks coincide (Fenley's pumping clock is four hours). Strikingly, when the hospital's generator fails, the equal-weight best order does not change: 3,000 hospital users are few compared with 70,000 water customers, so equal weights keep the hospital third. That result is the analysis telling you that its weights are wrong for this purpose. EX036 asks you to weight hospital person-hours more heavily; under a plausible weighting the hospital moves to first once its generator fails. The decision belongs to the emergency cell; the specialist's contribution is to show the order implied by each defensible set of weights, and how one failed backup changes it.
8.10 Common errors and limits
- Treating a difficult case as if it had a harmless option and searching for it while clocks run out.
- Restoring by habit rather than by explicit, pre-agreed weights.
- Weighting all person-hours equally and calling it neutral.
- Trusting the supplier's channel during a supplier-caused incident.
- Letting a specialist, rather than the accountable executive, decide on extortion payment.
- Accusing individuals before evidence supports it.
- Making statements to one stakeholder that contradict statements to another.
The limit of analysis in difficult cases is time. The enumeration in Lab 8 takes milliseconds; agreeing the weights takes weeks of planning. Do that work before the event, rehearse it, and accept that during the event you will be choosing among imperfect options with incomplete information.
Exercises — chapter 8
EX036 — Weights change the order. Objective: see how value weights determine restoration. Inputs: Lab 8. Constraints: weight each hospital person-hour 20 times a home person-hour; leave others at 1. Steps: (1) modify the harm function; (2) report the best order with the 48-hour generator and after the generator fails at 2 hours; (3) explain the change and who should set the weight. Expected evidence: code change, outputs, paragraph. Rubric: 3 = with a working generator the best order is unchanged (510,000); after failure, Harbour moves first (best 751,000 weighted person-hours, Harbour > Hollin > Fenley > Port); explanation identifies the weight as a pre-agreed emergency-management judgement; 2 = numbers right, explanation thin; 1 = not run; 0 = none.
EX037 — Structure an extortion decision. Objective: support an executive decision without making it. Inputs: Merrow Health's patient-administration system encrypted; backups intact but a clean rebuild estimated at 9 days; attackers claim to hold patient data and offer a decryptor. Constraints: no legal conclusions; patient safety first. Steps: (1) list options; (2) for each, recovery timeline, safety and service consequences, legal questions for counsel, ethical considerations and uncertainties; (3) state what evidence would change the analysis. Expected evidence: decision brief. Rubric: 3 = options include rebuild without payment and partial measures, decryptor reliability treated as uncertain, data-extortion separated from recovery, sanctions and reporting questions for counsel, safety measures independent of the choice; 2 = one consideration missing; 1 = recommends payment or refusal as technical fact; 0 = none.
EX038 — Supplier-caused outage. Objective: decide under a trusted-supplier failure. Inputs: an update from Merrow Water's endpoint security supplier crashes HMIs at four works overnight; the supplier says it is a defect. Constraints: plants must keep producing safe water. Steps: (1) immediate actions; (2) recovery order across sites; (3) whether and how to block further updates; (4) changes to update policy afterwards, with their security cost. Expected evidence: plan. Rubric: 3 = manual operation first, supplier channel paused pending verification, recovery ordered by backup clocks, staged update rings afterwards with explicit acceptance of slower protection; 2 = no policy change; 1 = waits for the supplier; 0 = none.
EX039 — Insider communications. Objective: communicate an ambiguous insider case. Inputs: Hollin event with the corrected timeline. Constraints: no individual named; employment and contractual processes respected. Steps: draft (1) an internal message to staff; (2) a message to Corvane; (3) a statement for the resilience forum. Expected evidence: three short texts. Rubric: 3 = factual, no accusation, actions on access stated (named accounts, suspended shared account), Corvane asked for session records under contract, forum given useful protective information; 2 = one text over-claims; 1 = accusatory; 0 = none.
EX040 — Conflicting obligations. Objective: reconcile demands without deception. Inputs: during the Hollin investigation, police ask Merrow Water not to disclose the vendor account detail; the regulator's notification deadline arrives; Merrow Energy asks whether it should suspend Corvane access; a journalist asks whether water was unsafe. Constraints: follow §8.7 principles; check with counsel where needed. Steps: for each party, what you say, what you withhold and why, and who approves. Expected evidence: table. Rubric: 3 = safety answer given truthfully (dose returned to normal within a minute and verified), regulator informed fully under confidentiality, Merrow Energy given protective advice without breaching police request (for example advice to review all shared vendor accounts), journalist not misled, approvals recorded; 2 = one party mishandled; 1 = deceptive statement; 0 = none.
Chapter synthesis
Difficult cases have harm in every option, under time pressure, with ambiguous evidence and conflicting duties. Prepare for them: map dependencies and backup clocks; agree restoration weights with emergency management in advance; plan for supplier-caused failures with update rings and independent recovery; structure extortion decisions for the accountable executive; act on access rather than on people in insider cases; and resolve conflicting obligations by never deceiving, putting safety first, withholding only with defensible reasons and recording each decision. Analysis is fast; agreement is slow — do the slow part before the event.
Continue studying — chapter 08 of 12
← Chapter 07 · Course chapter index · Chapter 09 →
Continue to the next authored lesson.