Lesson map
Lesson 03 published. Map, example, EX015.
flowchart TD
N0["Learning outcomes and connection"]
N1["3.1 What kind of contribution?"]
N2["3.2 Restoration ordering as a scheduling problem"]
N3["3.3 Dependency graphs with buffers"]
N4["3.4 The mechanism"]
N5["3.5 Hypotheses, metrics and the smallest effect of i"]
N6["3.6 Baselines"]
N7["3.7 Rival explanations"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX015"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["3.1 What kind of contribution?"]
N2["3.2 Restoration ordering as a scheduling problem"]
N3["3.3 Dependency graphs with buffers"]
N4["3.4 The mechanism"]
N5["3.5 Hypotheses, metrics and the smallest effect of i"]
N6["3.6 Baselines"]
N7["3.7 Rival explanations"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX015"]
[Learning outcomes and connection]
|
v
[3.1 What kind of contribution?]
|
v
[3.2 Restoration ordering as a scheduling problem]
|
v
[3.3 Dependency graphs with buffers]
|
v
[3.4 The mechanism]
|
v
[3.5 Hypotheses, metrics and the smallest effect of i]
|
v
[3.6 Baselines]
|
v
[3.7 Rival explanations]
|
v
[Worked example] --> [Exercise EX015]
Learning outcomes and connection
By the end of this chapter the learner will be able to:
- Type a doctoral contribution (artifact, method, measurement, empirical finding, theory) and state the evidence each type requires.
- Specify a mechanism as a chain of links that predicts a pattern of results, including where the intervention should not help.
- Represent cross-sector dependencies as a graph with buffers, compute time-to-impact, and explain what the representation leaves out.
- Relate restoration ordering to classical scheduling theory and use that relation to choose baselines and predict trade-offs.
- Write primary, mechanism, contribution-over-best-practice and non-inferiority hypotheses with a justified smallest effect of interest.
- Build a rival-explanation table and a set of predeclared failure conditions that would narrow or abandon the claim.
Chapter 2 identified the gap: dependency-aware restoration ordering is often proposed and weakly tested. This chapter turns it into a proposal that could be wrong. Chapter 4 will replicate the baselines and pilot the procedure.
3.1 What kind of contribution?
Lesson table; scroll horizontally to read all columns →
| Type | Example in CI security | Minimum evidence |
|---|---|---|
| Artifact | An open co-simulation of distribution, water and telecom | Verification tests; documentation; evidence of use by others |
| Method | A procedure for computing restoration priorities from dependency data | Comparison against the strongest current practice on stated metrics, in more than one context |
| Measurement | A validated instrument for buffer duration or dependency-map accuracy | Reliability and validity evidence |
| Empirical finding | "Operators already prioritize critical facilities but do not use buffer timing" | A sampled, documented study of operators |
| Theory | A mechanism explaining when dependency-aware ordering helps | Predictions that discriminate it from rivals, tested |
The Cascade Study's intended contribution is a method with a mechanism: a rule for ordering restoration (the artifact is incidental) and an account of when it outperforms current practice. Saying this early matters, because "we built a simulator and our method won" is an artifact demonstration, not a method contribution.
3.2 Restoration ordering as a scheduling problem
When a single restoration crew must restore feeders one at a time, restoration ordering is a single-machine scheduling problem: each feeder j is a job with processing time p_j (manual switching, inspection, re-energization) and the objective depends on what is being minimized.
- If the objective is customer-hours lost, each feeder contributes (customers on j) × (completion time C_j). Minimizing Σ w_j C_j on one machine is solved exactly by ordering jobs by w_j / p_j in decreasing order — Smith's rule (Smith, 1956). With equal expected restoration times, this is simply "most customers first". The conventional ordering used as a baseline in this book is therefore not naive: it is optimal for customer-hours in this idealized setting.
- If the objective is critical-service loss, a service s fed by feeder j is lost only from the moment its buffer b_s is exhausted until the feeder is restored: loss_s = w_s × max(0, C_j + lag − b_s). The buffer acts like a due date, and the objective resembles weighted tardiness. Single-machine total weighted tardiness is strongly NP-hard (Lenstra, Rinnooy Kan and Brucker, 1977), so practical rules are heuristics. Ordering by earliest due date — here, earliest time-to-impact — is a classical heuristic that ignores weights.
This relation gives the study three things a purely security framing would miss. First, a theoretical prediction of the trade-off: any ordering that departs from Smith's rule must increase expected customer-hours, so a non-inferiority margin on customer-hours is not optional. Second, an upper bound on achievable benefit: with few feeders, exhaustive enumeration gives an oracle ordering for each scenario draw, which shows how much room any heuristic leaves. Third, a known failure mode of the proposed heuristic: an earliest-time-to-impact rule ignores weights and fan-out, so it can lose to a rule that restores the feeder protecting the most weighted services, even if that feeder's first impact is slightly later. Chapter 6 shows exactly this.
3.3 Dependency graphs with buffers
3.3.1 Representation
Represent each supplier → dependant relation as a directed edge labelled with the dependant's buffer for that supplier. If supplier A fails at time t_A, dependant B fails at t_A + b_AB unless restored first. If B fails when any of its suppliers fails (an "OR" of failures — no redundancy), B's failure time is the minimum over incoming paths, which is a shortest-path computation with buffers as edge weights.
3.3.2 Time-to-impact lab
# Lab 3 - time-to-impact on a synthetic Riverbend dependency graph (offline, standard library only)
# Edge (supplier -> dependant, buffer_hours): the dependant fails `buffer_hours` after it loses that supplier.
import heapq
edges = [
("F2", "TEL", 8), # telecom central office: batteries plus generator fuel on hand
("F2", "HOSP", 26), # hospital: generator fuel on hand
("F3", "WATER", 9), # treatment plant generator + clearwell storage
("F3", "WSCADA", 2), # water SCADA server UPS
("TEL", "WSCADA", 0), # remote telemetry needs the carrier circuit
("WATER", "HOSP", 4), # hospital on-site water storage
("TEL", "DISPATCH", 1), # emergency dispatch: ring-down backup lasts 1 h
("F1", "RES", 0), ("F4", "COM", 0),
]
def time_to_impact(initial): # initial: {node: failure time}
graph = {}
for s, d, b in edges:
graph.setdefault(s, []).append((d, b))
best = dict(initial)
pq = [(t, n) for n, t in initial.items()]
heapq.heapify(pq)
while pq:
t, n = heapq.heappop(pq)
if t > best.get(n, float("inf")):
continue
for d, b in graph.get(n, []):
if t + b < best.get(d, float("inf")):
best[d] = t + b
heapq.heappush(pq, (t + b, d))
return best
tti = time_to_impact({"F1": 0, "F2": 0, "F3": 0, "F4": 0})
for node in sorted(tti, key=lambda n: (tti[n], n)):
print(f"{node:9} fails at t={tti[node]:>3} h")
degree = {}
for s, d, _ in edges:
degree[s] = degree.get(s, 0) + 1
degree[d] = degree.get(d, 0) + 1
print("degree ranking:", sorted(degree, key=lambda n: (-degree[n], n))[:4])
Expected output:
COM fails at t= 0 h
F1 fails at t= 0 h
F2 fails at t= 0 h
F3 fails at t= 0 h
F4 fails at t= 0 h
RES fails at t= 0 h
WSCADA fails at t= 2 h
TEL fails at t= 8 h
DISPATCH fails at t= 9 h
WATER fails at t= 9 h
HOSP fails at t= 13 h
degree ranking: ['TEL', 'F2', 'F3', 'HOSP']
Three lessons. Indirect paths dominate: the hospital's own generator gives 26 hours, but it fails at hour 13 through the water path (9 + 4). A study that recorded only each facility's direct power buffer would misjudge its priority by half a day. Degree is not consequence: degree centrality ranks TEL first, which is reasonable here, but it ranks HOSP above WATER and says nothing about timing. The OR assumption is strong: water SCADA fails at hour 0 + 0 via TEL only if telemetry has no alternative path; in many utilities operators can run the plant locally without SCADA, so WSCADA's failure may have no service consequence. The graph must distinguish loss of a function from loss of a service.
3.3.3 What the representation leaves out
Edges with fixed buffers omit: depletion dynamics (a generator's fuel lasts longer at partial load); redundancy ("AND" logic — a hospital with two independent feeds fails only when both fail); restoration dependencies (a feeder cannot be re-energized without communications to field crews — a dependency in the recovery direction); human buffers (manual operation, mutual aid, fuel deliveries that extend buffers if roads and telecom work); and adversary adaptation (an adversary who knows the priority list can target its top). A proposal must list which omissions could reverse its conclusions and how the study will test them (§3.7).
3.4 The mechanism
A mechanism is a chain of links, each of which could fail, that predicts a pattern, not only an average effect:
- Critical services draw on buffers whose durations differ widely (hours for telecom batteries and dispatch backups, days for hospital fuel).
- In a multi-feeder outage requiring sequential manual restoration, the order of restoration determines which buffers are exhausted.
- Conventional practice either maximizes customers restored (Smith's rule) or restores "critical customers" first in an order that does not use buffer timing.
- Ordering by time-to-impact restores the feeders whose critical services would fail first, so fewer buffers are exhausted.
- Therefore weighted critical-service loss falls.
The mechanism predicts a pattern:
- Benefit over customer-first ordering whenever critical services sit on feeders with few customers.
- Benefit over critical-flag ordering only when they disagree — when the buffer-time order among critical feeders differs from the customer-count order among them. When the orders coincide, the two rules are identical and the benefit must be zero.
- No benefit when buffers exceed total restoration time — if every buffer outlasts the outage, ordering cannot matter for critical services.
- A cost in customer-hours relative to Smith's rule, larger when critical services sit on small feeders.
- Fragility to map error — if the dependency map or buffers are wrong, the ordering can be worse than a simple flag list.
A mechanism that predicts where the intervention fails is worth more than one that predicts only success, because the failures are what distinguish it from rivals.
3.5 Hypotheses, metrics and the smallest effect of interest
Primary metric: weighted critical-service loss over a 72-hour window, with predeclared weights (hospital 5, water 4, telecom 3, emergency dispatch 4 per service-hour), elicited from the partners' emergency-management staff before any simulation (Chapter 1, §1.6.3). Secondary (safety) metric: customer-hours lost (thousands).
- H1 (benefit over customer-first). DEP reduces mean weighted critical-service loss by at least 20% compared with SILO.
- H2 (contribution over best practice). DEP reduces mean weighted critical-service loss by at least 10% compared with CRITFLAG (critical-customer flag list ordered by customers), in each study region.
- H3 (mechanism pattern). The DEP − CRITFLAG difference is zero when the two orders coincide and grows with the number of positions in which they differ; it shrinks toward zero as buffers grow relative to restoration time.
- H4a/H4b (non-inferiority). DEP increases mean customer-hours lost by no more than 15% compared with SILO (H4a) and with CRITFLAG (H4b).
Smallest effects of interest. The 20% and 10% thresholds and the 15% margin are practical judgments, not statistical conventions: partners stated (synthetic) that a change in restoration practice requires retraining, procedure revision and regulator explanation, which they would undertake only for a reduction of at least 10% in critical-service loss relative to their current critical-flag practice; and the cooperative's board would not accept more than 15% additional customer-hours without public consultation. Record who set each threshold and when; a threshold chosen after seeing results is worthless.
Note that H4a is predicted to fail by scheduling theory whenever critical services sit on small feeders. Predeclaring it anyway is correct: the study should report the size of the trade-off honestly rather than hide it, and a failed non-inferiority test is a finding for policy-makers.
3.6 Baselines
A contribution is only as strong as the baseline it beats.
Lesson table; scroll horizontally to read all columns →
| Baseline | Why include it | What beating it would show |
|---|---|---|
| SILO — most customers first | Optimal for customer-hours under equal durations (Smith's rule); common default | That criticality-aware ordering protects critical services — expected and weak as a contribution |
| CRITFLAG — feeders with any listed critical facility first, by customers | Represents widespread practice of maintaining critical-customer lists | The actual contribution: that timing information adds value beyond a list |
| ORACLE — best of all orderings for each draw (computable for ≤ 6–7 feeders) | Upper bound on any rule | How much room is left; whether DEP is near-optimal |
| RANDOM (optional) | Sanity check | That the metric is not degenerate |
Beating SILO is necessary but not sufficient; the claim lives or dies against CRITFLAG. The worked example in §3.9 shows how a proposal that omitted CRITFLAG would have looked impressive and been wrong.
3.7 Rival explanations
Lesson table; scroll horizontally to read all columns →
| Rival | Prediction that would support it | Design feature that tests it |
|---|---|---|
| Any criticality list explains the gain, not timing | DEP ≈ CRITFLAG | CRITFLAG baseline; H2 |
| The gain is an artifact of weights | Gain disappears with equal or alternative weights | Weight sensitivity (Chapter 6) |
| The gain is an artifact of short buffers in the chosen region | Gain vanishes with realistic longer buffers | Buffer-scaling analysis; second region with fuel contracts |
| The model was built to favor DEP (circularity) | Gain shrinks in an independently built model or with a wrong map | Second region built by a different modeller from partner data; map-error test |
| Operators would not follow the ordering | Tabletop choices differ from DEP even with time-to-impact display | Blinded operator tabletop (Chapter 5) |
| One-crew assumption drives the result | Gain disappears with several crews | Port Halvard with two crews |
| Discretization or implementation error | Results change with time step or implementation | Verification against an independent event-based implementation (Chapter 4) |
3.8 Failure conditions and risk register
Predeclared failure conditions (each maps to a claim change):
Lesson table; scroll horizontally to read all columns →
| Condition | Consequence for the claim |
|---|---|
| H2 fails in both regions | Abandon the method contribution; report that timing adds nothing measurable over critical-flag lists; retain the scheduling analysis as a theoretical contribution. |
| H2 fails in one region and holds in the other | Narrow to the conditions that distinguish them; test H3 to see if the mechanism explains the difference. |
| H3 pattern absent (gain unrelated to order divergence) | Mechanism unsupported; any gain is unexplained and must not be generalized. |
| H4b fails | Report a trade-off; the method is not "free" relative to current practice. |
| Gain collapses under plausible map errors | Claim conditional on map accuracy; add map-audit as a prerequisite. |
| Operators reject the ordering in tabletop | Claim limited to "a decision aid whose adoption is unproven"; study adoption barriers. |
Risk register (selection): partner withdrawal (mitigation: synthetic-region design that does not depend on one partner's data); sensitive data exposure (tiered storage, no real maps in public text); simulator error (independent re-implementation and unit tests); tabletop participants identifiable (aggregate reporting, no role-plus-utility combinations in public text); funder pressure (publication agreement signed before work starts).
3.9 Worked example: the proposal that nearly omitted the real baseline
The candidate's first draft compared DEP with SILO only and reported, in a pilot run, an almost 90% reduction in weighted critical-service loss. The supervisory committee asked two questions: What do utilities actually do today? and What would a perfect ordering achieve? Interviews (synthetic) with three dispatchers revealed that all three utilities keep a critical-customer list and restore those feeders early. The candidate added CRITFLAG and ORACLE. The pilot then showed DEP and CRITFLAG nearly indistinguishable in Riverbend, both far from ORACLE. The committee's verdict: the 90% figure measured the value of having a critical-customer list, which is not new; the open question is whether timing adds value, and Riverbend alone — with only two critical feeders, ordered the same way by most rules — cannot answer it. The proposal was revised to predeclare H2 and H3, to choose a second region on stated design criteria (more critical feeders, divergent buffer and customer orders, two crews, fuel contracts) before any simulation there, and to run a many-region analysis in which the divergence between orders varies systematically.
This is the single most important design decision in the book: choose the comparison that represents best current practice, not the one that makes the method look best.
3.10 Common errors and limits
- Comparing against a straw-man baseline (no prioritization at all, or random).
- Omitting the cost side of a trade-off that scheduling theory predicts.
- Defining "critical" after seeing which facilities benefit.
- Encoding buffers as point values without uncertainty.
- Treating a graph with OR semantics as if it modelled redundancy.
- Writing a mechanism that predicts only success.
-
Setting the smallest effect of interest by statistical convention instead of practical consequence.
-
Choosing the second context after seeing which kind of region favours the method, rather than fixing selection criteria in advance.
- Treating an earliest-time-to-impact rule as optimal when scheduling theory shows the weighted problem is hard and the rule ignores weights and fan-out.
- Recording only each facility's direct power buffer, so that indirect paths such as the hospital's dependence on water are missed.
- Letting the partner who supplies the data also choose the thresholds without recording who set them and when.
Exercises — chapter 3
EX015 — Contribution typing. Objective: match claims to evidence. Inputs: four synthetic thesis claims: (a) "We release GridWater-Sim 2.0"; (b) "Our buffer-duration survey instrument is reliable"; (c) "Utilities rarely use buffer timing when ordering restoration"; (d) "Time-to-impact ordering reduces critical-service loss." Constraints: use §3.1. Steps: (1) assign each claim a contribution type; (2) state the minimum evidence that type requires; (3) state which claim depends on another; (4) say which claim the Cascade Study must not make from (a) alone. Expected evidence: table. Rubric: 3 = correct types and evidence, (d) dependent on (a) only as instrument, (c) needs sampled empirical study; 2 = one error; 1 = generic evidence; 0 = none.
EX016 — Scheduling theory check. Objective: apply Smith's rule. Inputs: Riverbend customers F1 9000, F2 1200, F3 2500, F4 6000; restoration durations 4, 3, 5 and 4 hours respectively; one crew. Constraints: show calculations. Steps: (1) compute w/p for each feeder; (2) give Smith's order; (3) compute customer-hours (thousands) for Smith's order and for the order F2, F3, F1, F4; (4) state the percentage increase. Expected evidence: calculations. Rubric: 3 = order F1, F4, F3, F2 (ratios 2250, 1500, 500, 400), totals 135.7 and 227.6 thousand customer-hours, an increase of 67.7%; 2 = arithmetic slip; 1 = wrong rule; 0 = none.
EX017 — Time-to-impact extension. Objective: extend Lab 3. Inputs: Lab 3. Constraints: offline. Steps: (1) add a fuel-delivery dependency: the hospital's generator buffer becomes 60 h if TEL is available for ordering fuel, modelled by replacing the F2 → HOSP buffer with 60 and adding TEL → HOSP with buffer 30; (2) run and record HOSP's failure time; (3) explain why the result did not change; (4) state what the OR-semantics graph cannot express about this fuel arrangement. Expected evidence: code, output, explanation. Rubric: 3 = HOSP still 13 h via water; explains that a buffer extension conditional on another service is an AND/conditional relation the OR graph cannot express; 2 = output right, explanation partial; 1 = wrong output; 0 = not run.
EX018 — Hypothesis set. Objective: operationalize hypotheses. Inputs: the mechanism in §3.4, applied to a different domain of your choice (e.g., restoring telecom sites after a cyber incident at a carrier, with hospitals and dispatch as dependants). Constraints: one benefit hypothesis, one contribution-over-best-practice hypothesis, one mechanism-pattern hypothesis and one non-inferiority hypothesis; justified thresholds. Steps: (1) write each hypothesis with population, intervention, comparator, outcome and direction; (2) define each outcome operationally; (3) justify each threshold by a practical consequence. Expected evidence: hypothesis block. Rubric: 3 = complete and operational, best-practice comparator named, thresholds justified by consequences and owners; 2 = one element vague; 1 = thresholds arbitrary; 0 = none.
EX019 — Rival-explanation table. Objective: design against alternatives. Inputs: your EX018 hypotheses. Constraints: at least six rivals. Steps: (1) list at least six rival explanations for a positive result; (2) for each, state the prediction that would support it; (3) name the design feature that tests it. Expected evidence: table. Rubric: 3 = six or more plausible rivals including baseline adequacy, weights, circularity and adoption, each with a concrete design feature; 2 = four or five; 1 = generic; 0 = none.
EX020 — Failure conditions and second-context criteria. Objective: make the contribution falsifiable and avoid cherry-picking the validation context. Inputs: your proposal. Constraints: at least five failure conditions with consequences; selection criteria for a second context written before it is modelled. Steps: (1) write failure conditions mapped to claim changes; (2) write three to five context-selection criteria derived from the mechanism; (3) explain how a reviewer could verify the criteria were fixed in advance. Expected evidence: two lists and a paragraph. Rubric: 3 = each condition changes a specific claim, criteria derived from the mechanism (order divergence, crew count, buffer range), timestamped preregistration named; 2 = criteria not mechanism-derived; 1 = vague; 0 = none.
Chapter synthesis
A doctoral contribution must say what kind it is and must beat the strongest current practice, not a straw man. Restoration ordering is a scheduling problem: Smith's rule makes "most customers first" optimal for customer-hours, while buffer-limited critical services create a weighted-tardiness objective for which simple rules are heuristics. That theory predicts the trade-off, provides an oracle bound and exposes the proposed rule's weakness. Dependency graphs with buffers reveal indirect paths that dominate timing, but their OR semantics, fixed buffers and missing human and recovery-direction dependencies must be named. The mechanism predicts a pattern — benefit only where timing and customer counts disagree, none when buffers are long, a cost in customer-hours and fragility to map error — and the hypotheses, baselines, rivals and failure conditions are written so that each part of that pattern could be shown false.