OpenCyber Academy
Doctoral research · Chapter 3 · English
Course code: OCA-207Chapter 03: OCA-207-C003

Chapter 3 — Original contribution proposal

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 03 published. Map, example, EX015.

flowchart TD
  N0["Learning outcomes and connection"]
  N1["3.1 What kind of contribution?"]
  N2["3.2 Restoration ordering as a scheduling problem"]
  N3["3.3 Dependency graphs with buffers"]
  N4["3.4 The mechanism"]
  N5["3.5 Hypotheses, metrics and the smallest effect of i"]
  N6["3.6 Baselines"]
  N7["3.7 Rival explanations"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX015"]

Logical drawing

flowchart LR
  N0["Learning outcomes and connection"]
  N1["3.1 What kind of contribution?"]
  N2["3.2 Restoration ordering as a scheduling problem"]
  N3["3.3 Dependency graphs with buffers"]
  N4["3.4 The mechanism"]
  N5["3.5 Hypotheses, metrics and the smallest effect of i"]
  N6["3.6 Baselines"]
  N7["3.7 Rival explanations"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX015"]
[Learning outcomes and connection]
    |
    v
[3.1 What kind of contribution?]
    |
    v
[3.2 Restoration ordering as a scheduling problem]
    |
    v
[3.3 Dependency graphs with buffers]
    |
    v
[3.4 The mechanism]
    |
    v
[3.5 Hypotheses, metrics and the smallest effect of i]
    |
    v
[3.6 Baselines]
    |
    v
[3.7 Rival explanations]
    |
    v
[Worked example] --> [Exercise EX015]

Learning outcomes and connection

By the end of this chapter the learner will be able to:

  1. Type a doctoral contribution (artifact, method, measurement, empirical finding, theory) and state the evidence each type requires.
  2. Specify a mechanism as a chain of links that predicts a pattern of results, including where the intervention should not help.
  3. Represent cross-sector dependencies as a graph with buffers, compute time-to-impact, and explain what the representation leaves out.
  4. Relate restoration ordering to classical scheduling theory and use that relation to choose baselines and predict trade-offs.
  5. Write primary, mechanism, contribution-over-best-practice and non-inferiority hypotheses with a justified smallest effect of interest.
  6. Build a rival-explanation table and a set of predeclared failure conditions that would narrow or abandon the claim.

Chapter 2 identified the gap: dependency-aware restoration ordering is often proposed and weakly tested. This chapter turns it into a proposal that could be wrong. Chapter 4 will replicate the baselines and pilot the procedure.

3.1 What kind of contribution?

Lesson table; scroll horizontally to read all columns →

Type Example in CI security Minimum evidence
Artifact An open co-simulation of distribution, water and telecom Verification tests; documentation; evidence of use by others
Method A procedure for computing restoration priorities from dependency data Comparison against the strongest current practice on stated metrics, in more than one context
Measurement A validated instrument for buffer duration or dependency-map accuracy Reliability and validity evidence
Empirical finding "Operators already prioritize critical facilities but do not use buffer timing" A sampled, documented study of operators
Theory A mechanism explaining when dependency-aware ordering helps Predictions that discriminate it from rivals, tested

The Cascade Study's intended contribution is a method with a mechanism: a rule for ordering restoration (the artifact is incidental) and an account of when it outperforms current practice. Saying this early matters, because "we built a simulator and our method won" is an artifact demonstration, not a method contribution.

3.2 Restoration ordering as a scheduling problem

When a single restoration crew must restore feeders one at a time, restoration ordering is a single-machine scheduling problem: each feeder j is a job with processing time p_j (manual switching, inspection, re-energization) and the objective depends on what is being minimized.

This relation gives the study three things a purely security framing would miss. First, a theoretical prediction of the trade-off: any ordering that departs from Smith's rule must increase expected customer-hours, so a non-inferiority margin on customer-hours is not optional. Second, an upper bound on achievable benefit: with few feeders, exhaustive enumeration gives an oracle ordering for each scenario draw, which shows how much room any heuristic leaves. Third, a known failure mode of the proposed heuristic: an earliest-time-to-impact rule ignores weights and fan-out, so it can lose to a rule that restores the feeder protecting the most weighted services, even if that feeder's first impact is slightly later. Chapter 6 shows exactly this.

3.3 Dependency graphs with buffers

3.3.1 Representation

Represent each supplier → dependant relation as a directed edge labelled with the dependant's buffer for that supplier. If supplier A fails at time t_A, dependant B fails at t_A + b_AB unless restored first. If B fails when any of its suppliers fails (an "OR" of failures — no redundancy), B's failure time is the minimum over incoming paths, which is a shortest-path computation with buffers as edge weights.

3.3.2 Time-to-impact lab

# Lab 3 - time-to-impact on a synthetic Riverbend dependency graph (offline, standard library only)
# Edge (supplier -> dependant, buffer_hours): the dependant fails `buffer_hours` after it loses that supplier.
import heapq
edges = [
    ("F2", "TEL", 8),     # telecom central office: batteries plus generator fuel on hand
    ("F2", "HOSP", 26),   # hospital: generator fuel on hand
    ("F3", "WATER", 9),   # treatment plant generator + clearwell storage
    ("F3", "WSCADA", 2),  # water SCADA server UPS
    ("TEL", "WSCADA", 0), # remote telemetry needs the carrier circuit
    ("WATER", "HOSP", 4), # hospital on-site water storage
    ("TEL", "DISPATCH", 1),  # emergency dispatch: ring-down backup lasts 1 h
    ("F1", "RES", 0), ("F4", "COM", 0),
]
def time_to_impact(initial):  # initial: {node: failure time}
    graph = {}
    for s, d, b in edges:
        graph.setdefault(s, []).append((d, b))
    best = dict(initial)
    pq = [(t, n) for n, t in initial.items()]
    heapq.heapify(pq)
    while pq:
        t, n = heapq.heappop(pq)
        if t > best.get(n, float("inf")):
            continue
        for d, b in graph.get(n, []):
            if t + b < best.get(d, float("inf")):
                best[d] = t + b
                heapq.heappush(pq, (t + b, d))
    return best
tti = time_to_impact({"F1": 0, "F2": 0, "F3": 0, "F4": 0})
for node in sorted(tti, key=lambda n: (tti[n], n)):
    print(f"{node:9} fails at t={tti[node]:>3} h")
degree = {}
for s, d, _ in edges:
    degree[s] = degree.get(s, 0) + 1
    degree[d] = degree.get(d, 0) + 1
print("degree ranking:", sorted(degree, key=lambda n: (-degree[n], n))[:4])

Expected output:

COM       fails at t=  0 h
F1        fails at t=  0 h
F2        fails at t=  0 h
F3        fails at t=  0 h
F4        fails at t=  0 h
RES       fails at t=  0 h
WSCADA    fails at t=  2 h
TEL       fails at t=  8 h
DISPATCH  fails at t=  9 h
WATER     fails at t=  9 h
HOSP      fails at t= 13 h
degree ranking: ['TEL', 'F2', 'F3', 'HOSP']

Three lessons. Indirect paths dominate: the hospital's own generator gives 26 hours, but it fails at hour 13 through the water path (9 + 4). A study that recorded only each facility's direct power buffer would misjudge its priority by half a day. Degree is not consequence: degree centrality ranks TEL first, which is reasonable here, but it ranks HOSP above WATER and says nothing about timing. The OR assumption is strong: water SCADA fails at hour 0 + 0 via TEL only if telemetry has no alternative path; in many utilities operators can run the plant locally without SCADA, so WSCADA's failure may have no service consequence. The graph must distinguish loss of a function from loss of a service.

3.3.3 What the representation leaves out

Edges with fixed buffers omit: depletion dynamics (a generator's fuel lasts longer at partial load); redundancy ("AND" logic — a hospital with two independent feeds fails only when both fail); restoration dependencies (a feeder cannot be re-energized without communications to field crews — a dependency in the recovery direction); human buffers (manual operation, mutual aid, fuel deliveries that extend buffers if roads and telecom work); and adversary adaptation (an adversary who knows the priority list can target its top). A proposal must list which omissions could reverse its conclusions and how the study will test them (§3.7).

3.4 The mechanism

A mechanism is a chain of links, each of which could fail, that predicts a pattern, not only an average effect:

  1. Critical services draw on buffers whose durations differ widely (hours for telecom batteries and dispatch backups, days for hospital fuel).
  2. In a multi-feeder outage requiring sequential manual restoration, the order of restoration determines which buffers are exhausted.
  3. Conventional practice either maximizes customers restored (Smith's rule) or restores "critical customers" first in an order that does not use buffer timing.
  4. Ordering by time-to-impact restores the feeders whose critical services would fail first, so fewer buffers are exhausted.
  5. Therefore weighted critical-service loss falls.

The mechanism predicts a pattern:

A mechanism that predicts where the intervention fails is worth more than one that predicts only success, because the failures are what distinguish it from rivals.

3.5 Hypotheses, metrics and the smallest effect of interest

Primary metric: weighted critical-service loss over a 72-hour window, with predeclared weights (hospital 5, water 4, telecom 3, emergency dispatch 4 per service-hour), elicited from the partners' emergency-management staff before any simulation (Chapter 1, §1.6.3). Secondary (safety) metric: customer-hours lost (thousands).

Smallest effects of interest. The 20% and 10% thresholds and the 15% margin are practical judgments, not statistical conventions: partners stated (synthetic) that a change in restoration practice requires retraining, procedure revision and regulator explanation, which they would undertake only for a reduction of at least 10% in critical-service loss relative to their current critical-flag practice; and the cooperative's board would not accept more than 15% additional customer-hours without public consultation. Record who set each threshold and when; a threshold chosen after seeing results is worthless.

Note that H4a is predicted to fail by scheduling theory whenever critical services sit on small feeders. Predeclaring it anyway is correct: the study should report the size of the trade-off honestly rather than hide it, and a failed non-inferiority test is a finding for policy-makers.

3.6 Baselines

A contribution is only as strong as the baseline it beats.

Lesson table; scroll horizontally to read all columns →

Baseline Why include it What beating it would show
SILO — most customers first Optimal for customer-hours under equal durations (Smith's rule); common default That criticality-aware ordering protects critical services — expected and weak as a contribution
CRITFLAG — feeders with any listed critical facility first, by customers Represents widespread practice of maintaining critical-customer lists The actual contribution: that timing information adds value beyond a list
ORACLE — best of all orderings for each draw (computable for ≤ 6–7 feeders) Upper bound on any rule How much room is left; whether DEP is near-optimal
RANDOM (optional) Sanity check That the metric is not degenerate

Beating SILO is necessary but not sufficient; the claim lives or dies against CRITFLAG. The worked example in §3.9 shows how a proposal that omitted CRITFLAG would have looked impressive and been wrong.

3.7 Rival explanations

Lesson table; scroll horizontally to read all columns →

Rival Prediction that would support it Design feature that tests it
Any criticality list explains the gain, not timing DEP ≈ CRITFLAG CRITFLAG baseline; H2
The gain is an artifact of weights Gain disappears with equal or alternative weights Weight sensitivity (Chapter 6)
The gain is an artifact of short buffers in the chosen region Gain vanishes with realistic longer buffers Buffer-scaling analysis; second region with fuel contracts
The model was built to favor DEP (circularity) Gain shrinks in an independently built model or with a wrong map Second region built by a different modeller from partner data; map-error test
Operators would not follow the ordering Tabletop choices differ from DEP even with time-to-impact display Blinded operator tabletop (Chapter 5)
One-crew assumption drives the result Gain disappears with several crews Port Halvard with two crews
Discretization or implementation error Results change with time step or implementation Verification against an independent event-based implementation (Chapter 4)

3.8 Failure conditions and risk register

Predeclared failure conditions (each maps to a claim change):

Lesson table; scroll horizontally to read all columns →

Condition Consequence for the claim
H2 fails in both regions Abandon the method contribution; report that timing adds nothing measurable over critical-flag lists; retain the scheduling analysis as a theoretical contribution.
H2 fails in one region and holds in the other Narrow to the conditions that distinguish them; test H3 to see if the mechanism explains the difference.
H3 pattern absent (gain unrelated to order divergence) Mechanism unsupported; any gain is unexplained and must not be generalized.
H4b fails Report a trade-off; the method is not "free" relative to current practice.
Gain collapses under plausible map errors Claim conditional on map accuracy; add map-audit as a prerequisite.
Operators reject the ordering in tabletop Claim limited to "a decision aid whose adoption is unproven"; study adoption barriers.

Risk register (selection): partner withdrawal (mitigation: synthetic-region design that does not depend on one partner's data); sensitive data exposure (tiered storage, no real maps in public text); simulator error (independent re-implementation and unit tests); tabletop participants identifiable (aggregate reporting, no role-plus-utility combinations in public text); funder pressure (publication agreement signed before work starts).

3.9 Worked example: the proposal that nearly omitted the real baseline

The candidate's first draft compared DEP with SILO only and reported, in a pilot run, an almost 90% reduction in weighted critical-service loss. The supervisory committee asked two questions: What do utilities actually do today? and What would a perfect ordering achieve? Interviews (synthetic) with three dispatchers revealed that all three utilities keep a critical-customer list and restore those feeders early. The candidate added CRITFLAG and ORACLE. The pilot then showed DEP and CRITFLAG nearly indistinguishable in Riverbend, both far from ORACLE. The committee's verdict: the 90% figure measured the value of having a critical-customer list, which is not new; the open question is whether timing adds value, and Riverbend alone — with only two critical feeders, ordered the same way by most rules — cannot answer it. The proposal was revised to predeclare H2 and H3, to choose a second region on stated design criteria (more critical feeders, divergent buffer and customer orders, two crews, fuel contracts) before any simulation there, and to run a many-region analysis in which the divergence between orders varies systematically.

This is the single most important design decision in the book: choose the comparison that represents best current practice, not the one that makes the method look best.

3.10 Common errors and limits

Exercises — chapter 3

EX015 — Contribution typing. Objective: match claims to evidence. Inputs: four synthetic thesis claims: (a) "We release GridWater-Sim 2.0"; (b) "Our buffer-duration survey instrument is reliable"; (c) "Utilities rarely use buffer timing when ordering restoration"; (d) "Time-to-impact ordering reduces critical-service loss." Constraints: use §3.1. Steps: (1) assign each claim a contribution type; (2) state the minimum evidence that type requires; (3) state which claim depends on another; (4) say which claim the Cascade Study must not make from (a) alone. Expected evidence: table. Rubric: 3 = correct types and evidence, (d) dependent on (a) only as instrument, (c) needs sampled empirical study; 2 = one error; 1 = generic evidence; 0 = none.

EX016 — Scheduling theory check. Objective: apply Smith's rule. Inputs: Riverbend customers F1 9000, F2 1200, F3 2500, F4 6000; restoration durations 4, 3, 5 and 4 hours respectively; one crew. Constraints: show calculations. Steps: (1) compute w/p for each feeder; (2) give Smith's order; (3) compute customer-hours (thousands) for Smith's order and for the order F2, F3, F1, F4; (4) state the percentage increase. Expected evidence: calculations. Rubric: 3 = order F1, F4, F3, F2 (ratios 2250, 1500, 500, 400), totals 135.7 and 227.6 thousand customer-hours, an increase of 67.7%; 2 = arithmetic slip; 1 = wrong rule; 0 = none.

EX017 — Time-to-impact extension. Objective: extend Lab 3. Inputs: Lab 3. Constraints: offline. Steps: (1) add a fuel-delivery dependency: the hospital's generator buffer becomes 60 h if TEL is available for ordering fuel, modelled by replacing the F2 → HOSP buffer with 60 and adding TEL → HOSP with buffer 30; (2) run and record HOSP's failure time; (3) explain why the result did not change; (4) state what the OR-semantics graph cannot express about this fuel arrangement. Expected evidence: code, output, explanation. Rubric: 3 = HOSP still 13 h via water; explains that a buffer extension conditional on another service is an AND/conditional relation the OR graph cannot express; 2 = output right, explanation partial; 1 = wrong output; 0 = not run.

EX018 — Hypothesis set. Objective: operationalize hypotheses. Inputs: the mechanism in §3.4, applied to a different domain of your choice (e.g., restoring telecom sites after a cyber incident at a carrier, with hospitals and dispatch as dependants). Constraints: one benefit hypothesis, one contribution-over-best-practice hypothesis, one mechanism-pattern hypothesis and one non-inferiority hypothesis; justified thresholds. Steps: (1) write each hypothesis with population, intervention, comparator, outcome and direction; (2) define each outcome operationally; (3) justify each threshold by a practical consequence. Expected evidence: hypothesis block. Rubric: 3 = complete and operational, best-practice comparator named, thresholds justified by consequences and owners; 2 = one element vague; 1 = thresholds arbitrary; 0 = none.

EX019 — Rival-explanation table. Objective: design against alternatives. Inputs: your EX018 hypotheses. Constraints: at least six rivals. Steps: (1) list at least six rival explanations for a positive result; (2) for each, state the prediction that would support it; (3) name the design feature that tests it. Expected evidence: table. Rubric: 3 = six or more plausible rivals including baseline adequacy, weights, circularity and adoption, each with a concrete design feature; 2 = four or five; 1 = generic; 0 = none.

EX020 — Failure conditions and second-context criteria. Objective: make the contribution falsifiable and avoid cherry-picking the validation context. Inputs: your proposal. Constraints: at least five failure conditions with consequences; selection criteria for a second context written before it is modelled. Steps: (1) write failure conditions mapped to claim changes; (2) write three to five context-selection criteria derived from the mechanism; (3) explain how a reviewer could verify the criteria were fixed in advance. Expected evidence: two lists and a paragraph. Rubric: 3 = each condition changes a specific claim, criteria derived from the mechanism (order divergence, crew count, buffer range), timestamped preregistration named; 2 = criteria not mechanism-derived; 1 = vague; 0 = none.

Chapter synthesis

A doctoral contribution must say what kind it is and must beat the strongest current practice, not a straw man. Restoration ordering is a scheduling problem: Smith's rule makes "most customers first" optimal for customer-hours, while buffer-limited critical services create a weighted-tardiness objective for which simple rules are heuristics. That theory predicts the trade-off, provides an oracle bound and exposes the proposed rule's weakness. Dependency graphs with buffers reveal indirect paths that dominate timing, but their OR semantics, fixed buffers and missing human and recovery-direction dependencies must be named. The mechanism predicts a pattern — benefit only where timing and customer counts disagree, none when buffers are long, a cost in customer-hours and fragility to map error — and the hypotheses, baselines, rivals and failure conditions are written so that each part of that pattern could be shown false.

Worked answers

EX015 — Contribution typing. (a) Artifact — needs documentation, verification tests and evidence of use. (b) Measurement — needs reliability (test–retest, inter-rater) and validity evidence. (c) Empirical finding — needs a sampled, documented study of utilities, with sampling frame and response rates. (d) Method — needs comparison against the strongest current practice on stated metrics in more than one context. (d) uses (a) as an instrument; (a) alone cannot support (d), because the simulator's existence says nothing about whether the method beats current practice. (c) is a precondition for knowing whether (d) is novel in practice.

EX016 — Scheduling theory check. Ratios w/p: F1 9000/4 = 2250; F2 1200/3 = 400; F3 2500/5 = 500; F4 6000/4 = 1500. Smith's order: F1, F4, F3, F2. Completion times 4, 8, 13, 16 h → customer-hours = 9000·4 + 6000·8 + 2500·13 + 1200·16 = 135,700 → 135.7 thousand. Order F2, F3, F1, F4: completions 3, 8, 12, 16 → 1200·3 + 2500·8 + 9000·12 + 6000·16 = 227,600 → 227.6 thousand. Increase 67.7%. (In this book's labs, customer-hours are also counted per feeder until restoration; the restart lag applies only to critical services.)

EX017 — Time-to-impact extension. With F2 → HOSP = 60 and TEL → HOSP = 30, the hospital's failure time via power is min(60, 8 + 30 = 38) = 38 h, but via water it is 9 + 4 = 13 h, so HOSP still fails at 13 h. The result does not change because the binding path is water. More fundamentally, the arrangement "the generator lasts 60 h if telecom works to order fuel, otherwise 26 h" is a conditional buffer — an AND/OR combination that a shortest-path graph with fixed edge weights cannot express. Modelling it requires state-dependent buffers (as in Lab 5's dependency intervals, extended) or a simulation in which fuel deliveries are events that require TEL.

EX018 — Hypothesis set. Example for telecom-site restoration after a carrier cyber incident: Benefit — "Restoring cell sites in order of dependent-service time-to-impact reduces weighted loss of emergency-call capacity and hospital connectivity by ≥ 20% compared with restoring by subscribers served." Best practice — "…by ≥ 10% compared with the carrier's existing priority-site list." Mechanism — "The gain is zero where list order and time-to-impact order coincide and grows with divergence; it vanishes when site battery autonomy exceeds total restoration time." Non-inferiority — "Subscriber-hours lost increase by ≤ 15% compared with the priority-site list." Outcomes defined operationally (for example, emergency-call attempts that fail per hour, from network counters in a simulation). Thresholds justified by the carrier's stated change cost and the regulator's service-quality expectations, with owners and dates.

EX019 — Rival-explanation table. Rivals: any priority list explains the gain (test: list baseline); weights (test: alternative weights); short batteries in the chosen network (test: battery scaling, second network); circularity (test: independent model, map error); adoption (test: tabletop with engineers); single crew or travel ignored (test: crews and travel matrix); traffic rerouting to neighbouring sites makes the order irrelevant (test: model with neighbour coverage). Full credit includes at least circularity, baseline adequacy, weights and adoption.

EX020 — Failure conditions and second-context criteria. Failure conditions: as in §3.8, adapted. Second-context criteria derived from the mechanism: at least three critical-service sites; divergence between list order and time-to-impact order in at least two positions; buffer range overlapping total restoration time; a crew model different from the first context. Verification by reviewers: a timestamped preregistration amendment (in a public registry or a repository commit with a signed tag) dated before any simulation of the second context, plus the modeller's attestation that they had no access to first-context results.