◇ OpenCyber Academy
Doctoral research · Chapter 1 · English

Chapter 1 — Advanced research integrity and methods

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 1 — Advanced research integrity and methods

Lesson map

Lesson 01 published. Map, example, EX001.

flowchart TD
  N0["Learning outcomes and connection"]
  N1["1.1 What is being studied: services, functions and c"]
  N2["1.2 Evidence in infrastructure research"]
  N3["1.3 Integrity when partners are regulated operators"]
  N4["1.4 Safety, dual use and ethics"]
  N5["1.5 Protected information and data agreements"]
  N6["1.6 Measurement: resilience and consequence metrics"]
  N7["1.7 Common failure modes in infrastructure research"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX001"]

Logical drawing

flowchart LR
  N0["Learning outcomes and connection"]
  N1["1.1 What is being studied: services, functions and c"]
  N2["1.2 Evidence in infrastructure research"]
  N3["1.3 Integrity when partners are regulated operators"]
  N4["1.4 Safety, dual use and ethics"]
  N5["1.5 Protected information and data agreements"]
  N6["1.6 Measurement: resilience and consequence metrics"]
  N7["1.7 Common failure modes in infrastructure research"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX001"]
[Learning outcomes and connection]
    |
    v
[1.1 What is being studied: services, functions and c]
    |
    v
[1.2 Evidence in infrastructure research]
    |
    v
[1.3 Integrity when partners are regulated operators]
    |
    v
[1.4 Safety, dual use and ethics]
    |
    v
[1.5 Protected information and data agreements]
    |
    v
[1.6 Measurement: resilience and consequence metrics]
    |
    v
[1.7 Common failure modes in infrastructure research]
    |
    v
[Worked example] --> [Exercise EX001]

Learning outcomes and connection

By the end of this chapter the learner will be able to:

  1. Define critical infrastructure, critical function, dependency, interdependency and resilience precisely enough that a measurement can be attached to each, and explain why the definitions used by different governments are not interchangeable.
  2. Distinguish cyber observations from consequence claims and state what evidence is needed to move from one to the other.
  3. Apply research-integrity safeguards — pre-specification, deviation logs, separation of exploratory and confirmatory work, conflict-of-interest disclosure — to studies whose partners are regulated operators.
  4. Identify the safety, dual-use, legal and information-protection hazards of infrastructure research and design proportionate safeguards, using the Menlo Report principles as a working structure.
  5. Choose and defend resilience and consequence metrics, recognizing that metric choice can reverse the ranking of scenarios and that observation windows censor unrecovered outages.
  6. Write a research protocol that a hostile but fair reviewer — and a utility's safety officer — could audit, including stop conditions.

The chapter is the methodological contract for the rest of the book. Chapter 2 reads the literature with its evidence vocabulary; Chapter 3 uses its protocol template; Chapters 4–5 use its metrics and validity tools; Chapters 6–8 use its integrity, safety and information-protection standards to judge the finished work.

1.1 What is being studied: services, functions and consequences

1.1.1 Definitions that carry weight

"Critical infrastructure" is a legal and policy term before it is a technical one, and its definition decides what a study is allowed to generalize to. In the United States the statutory definition in the USA PATRIOT Act (42 U.S.C. § 5195c(e)) describes systems and assets "so vital to the United States that the incapacity or destruction of such systems and assets would have a debilitating impact on security, national economic security, national public health or safety". U.S. policy organizes those assets into sectors; the National Security Memorandum on Critical Infrastructure Security and Resilience (NSM-22), issued 30 April 2024, rescinded and replaced Presidential Policy Directive 21 (2013), retained the sector structure, named the Cybersecurity and Infrastructure Security Agency (CISA) as national coordinator and assigned Sector Risk Management Agencies (CISA overview; Congressional Research Service summary). The CRS summary notes that a later administration may rescind or modify NSM-22 without congressional action, so a thesis must cite the instrument in force on a stated date.

The European Union uses two coordinated instruments: Directive (EU) 2022/2557 on the resilience of critical entities (CER) for all-hazards resilience of designated critical entities and Directive (EU) 2022/2555 (NIS2) for cybersecurity of essential and important entities. A study that calls a hospital "critical infrastructure" in the U.S. sense and an "essential entity" in the NIS2 sense is not using synonyms: the populations differ, the obligations differ and the reporting thresholds differ.

A complementary lens is the function rather than the asset. CISA's National Critical Functions set, released in April 2019, lists 55 functions grouped as Connect, Distribute, Manage and Supply — for example, "Distribute electricity" or "Supply water". Functions cross sectors and organizations. For research, the functional lens has a decisive advantage: it names something whose loss can be measured (service delivered, people affected, duration) rather than something whose compromise can be observed (a host, a credential, a controller).

Working definitions used in this book:

Term Working definition What you must measure to use it
Critical service A service whose interruption beyond a stated duration causes harm to health, safety, security or essential economic activity for a stated population. Population served, harm threshold, tolerable outage duration.
Dependency A one-directional relation in which the state of service B depends on service A (Rinaldi, Peerenboom and Kelly, 2001). Direction, type, buffer, coupling strength.
Interdependency A bidirectional relation: each service's state depends on the other's, directly or through a chain (same source). The cycle and its timing.
Buffer Stored capacity that lets a dependant keep operating after losing a supplier: fuel, batteries, water storage, inventory, manual procedures. Duration under stated load, and how it is depleted.
Consequence The effect on people and on the functions they rely on, measured in service units (customer-hours, patient diversions, litres not delivered). A unit, a population, a time window.
Resilience The ability of a system to withstand, absorb, recover from and adapt to a disruption; operationalized by metrics over a performance curve (§1.6). A performance measure over time and a chosen summary.

Rinaldi, Peerenboom and Kelly classify interdependencies as physical (a material output of one is an input of another), cyber (dependence on information transmitted through an information infrastructure), geographic (a local event affects co-located components) and logical (any other mechanism, such as markets or policy). Their six "dimensions" — environment, coupling and response behavior, type of failure (cascading, escalating, common cause), infrastructure characteristics, state of operation and type of interdependency — remain a useful checklist for what a model ignores. A doctoral study should state which of these dimensions it models and which it holds fixed.

1.1.2 The cyber observation is not the consequence

The defining epistemic problem of the field is the gap between what can be observed about a cyber event and what happened to people. Consider three public examples, described only at the level their publishers released:

Each case links a cyber observation (malware, access, commands) to a consequence (outage, trip, avoided harm) through a chain of physical and organizational mechanisms. A research claim that skips the chain — "attacks on substations cause blackouts" — is not wrong so much as unsupported. Write the chain explicitly; every link is a place where a buffer, a manual procedure, a protection system or a human decision can break it.

1.1.3 The claim ladder for infrastructure research

The IR doctoral book in this series introduced a claim ladder (observation, inference, case claim, research claim). Infrastructure research needs one more rung, because the field's central claims are about consequences in physical systems that the researcher usually cannot observe directly:

Level Kind of statement Example (synthetic) Typical evidence
L1 Observation A named source recorded something. "REC's outage management system logged feeder F3 open at 02:10." Preserved record with clock state.
L2 Inference A conclusion from observations under stated assumptions. "F3 was opened by a remote command, not by protection." Command logs, relay event records, assumptions stated.
L3 Case claim A statement about what happened in one event. "The treatment plant ran on generator for 7 h, then lost pressure in the high zone." Converging records from several owners.
L4 Consequence claim A statement about effects on people or functions. "About 2,500 customers lacked potable water pressure for about 5 h." Service records, customer reports, field verification.
L5 Research claim A general statement about a population of systems or events. "Dependency-aware restoration reduces critical-service loss in distribution outages." Many cases, a validated model, or a designed study.

Moving up a rung always adds assumptions. The protocol in §1.8 requires every claim in a thesis to be tagged with its rung and its assumptions.

1.2 Evidence in infrastructure research

1.2.1 Five sources of evidence and their characteristic biases

  1. Incident records and public reports. Rare, selectively disclosed, written for purposes other than research, and shaped by legal exposure. Publication bias is extreme: events with dramatic consequences or attributable adversaries are reported; quietly absorbed disruptions are not. The population of incidents is unknown.
  2. Operational data from partners. SCADA historians, outage management records, work orders, alarm logs. High fidelity but owned by the operator, often protected by law or contract, and produced by systems tuned for operations rather than measurement. Sampling is decided by the partner's willingness, which correlates with maturity.
  3. Testbeds. Physical or hybrid laboratory plants such as the Secure Water Treatment (SWaT) testbed described by Goh, Adepu, Junejo and Mathur (2016). They allow controlled attacks and labeled data, at the cost of scale, age and realism: a six-stage teaching plant is not a city's water system.
  4. Models and simulations. Network, flow, input–output, agent-based, system-dynamics and co-simulation models (Chapter 2). They allow counterfactuals and scale but prove only what their structure and parameters permit; their validity must be established, not assumed (Sargent, 2013).
  5. People. Interviews, surveys, tabletop exercises and observation of operators, emergency managers and regulators. They reveal buffers that exist only as practice — manual workarounds, informal mutual aid — and also reveal what people say they would do, which is not what they do under stress.

Strong doctoral work triangulates: a model's prediction is checked against a historical record, a testbed experiment and the judgment of operators, and the thesis explains what each source can and cannot vouch for.

1.2.2 The base-rate and denominator problem

Infrastructure incidents with physical consequences are rare. A study that collects "cyber incidents affecting utilities" from public sources has a numerator without a denominator: it cannot say how often such incidents occur, only how often they are reported. Rates require a defined population (all utilities of a type in a jurisdiction), a defined period and a reporting mechanism with known completeness. Mandatory reporting regimes may eventually help: the U.S. Cyber Incident Reporting for Critical Infrastructure Act of 2022 (CIRCIA) requires CISA to issue rules; CISA's 2024 proposed rule would require reporting of covered incidents within 72 hours and of ransom payments within 24 hours, and the final rule was sent for interagency review on 1 October 2026 after missing earlier targets (Inside Privacy, 2026). Until a final rule is published and effective, and until its data are made available to researchers under some access regime, no thesis should treat reporting obligations as data.

1.2.3 Hindsight, counterfactuals and "near misses"

Consequence claims in this field are often counterfactual: "had the operators not switched to manual control, the outage would have lasted days." Counterfactuals are legitimate research objects only when they are produced by an explicit model whose assumptions are disclosed and whose sensitivity is reported. Hindsight makes the actual path look inevitable and the alternatives look obvious; Chapter 5's tabletop design shows how to present scenarios to participants without outcome knowledge.

1.3 Integrity when partners are regulated operators

1.3.1 Pressures specific to the field

Questionable research practices — selective reporting, flexible analysis, hypothesizing after results are known — are general (Simmons, Nelson and Simonsohn, 2011; Kerr, 1998). Infrastructure research adds four pressures:

1.3.2 Pre-specification for simulation and mixed-methods studies

Preregistration is not only for experiments with human participants (Nosek et al., 2018). For a simulation study, pre-specify: the model version and its verification tests; parameter distributions and their sources; the scenario families and how many draws of each; the comparison policies, including the strongest available baseline; the primary metric, the secondary or safety metric and any non-inferiority margin; the seeds; and the analysis (paired differences, interval method). For interviews and tabletops, pre-specify the sampling frame, the protocol, the coding frame's starting categories and the rule for adding categories. A deviation log records every change with date, reason and whether results had been seen.

1.3.3 Originality and provenance

Infrastructure literature is spread across engineering, operations research, economics, public policy and computer security, and the same idea is often rediscovered under different names: "dependency-aware restoration" in power engineering, "restoration prioritization" in emergency management, "service-level criticality" in IT. A doctoral originality claim therefore needs a search across disciplines (Chapter 2) and an honest statement of what is new: the method, the application, the evidence or the synthesis.

1.4 Safety, dual use and ethics

1.4.1 Why infrastructure research needs its own ethical analysis

The Menlo Report (Dittrich and Kenneally, 2012) adapted the Belmont principles — respect for persons, beneficence, justice — to information and communication technology research and added respect for law and public interest. Applying it to infrastructure research reveals stakeholders who are absent from most security studies: the population served by the infrastructure, who never consented and may be harmed by an experiment that goes wrong; the operator's staff, who may be exposed to blame; and third parties connected by interdependency.

Menlo principle Infrastructure-specific application
Respect for persons Informed consent for interviewed or observed staff; protection from disciplinary use of research data; separate consent for audio recording. The served population cannot consent, so their interests must be represented by risk minimization.
Beneficence No experiment on production control systems. Physical-safety review for any lab that energizes equipment. Weigh the benefit of publishing a vulnerability or weakness pattern against the uplift it gives to an adversary.
Justice Do not concentrate research burden (and risk of exposure) on small, under-resourced utilities that cannot say no to a funder; do not design restoration methods that systematically disadvantage a population without saying so (Chapter 3's non-inferiority margin).
Respect for law and public interest Comply with information-protection regimes (§1.5), export controls where relevant, computer-misuse law and partner contracts; be transparent about methods so the public interest in accountable research is served.

1.4.2 Hard rules used in this book

  1. Never test on production. No scanning, probing, fuzzing or command injection against any live operational system, including "just reading" with industrial protocols, without a written authorization from the asset owner that names scope, time, safety controls and a stop authority. NIST's Guide to Operational Technology Security, SP 800-82 Rev. 3 (September 2023) explains why OT differs from IT: availability and safety dominate, devices may fail under traffic that IT systems tolerate, and changes require engineering review. NIST released an initial public draft of Revision 4 in September 2026, with comments due 30 November 2026 (CSRC page); cite the revision you used.
  2. Simulate or emulate first. Use models, co-simulation, digital twins or isolated testbeds. When a physical testbed is used, it needs its own safety case (pressure, voltage, chemical dosing) reviewed by a qualified engineer.
  3. Separate capability from consequence in publication. A thesis may publish how a class of failure propagates and what buffers stop it; it should not publish an asset-specific target list or a step-by-step path to a physical effect.
  4. Have a stop authority. Any study involving a partner's staff or systems names a person on the partner side who can stop it instantly, and the protocol says what happens to data collected before the stop.

1.4.3 Dual-use review

A dual-use review asks three questions before any publication: Uplift — does this result make it materially easier for an adversary to cause harm, compared with what is already public? Defensive value — does it help defenders more than attackers, and do defenders have the means to act on it? Mitigation — can the harmful part be generalized, delayed (until fixes exist) or moved to controlled access without losing scientific value? Chapter 6 runs a full dual-use review of the Cascade Study's results; Chapter 8 places it in a coordinated-disclosure timeline.

1.5 Protected information and data agreements

1.5.1 Regimes the researcher will meet

Infrastructure data often fall under legal protection regimes separate from personal-data law. In the United States, examples include Protected Critical Infrastructure Information (PCII), Chemical-terrorism Vulnerability Information (CVI), Sensitive Security Information (SSI) in transportation and Critical Energy/Electric Infrastructure Information (CEII) at the Federal Energy Regulatory Commission. Their scope and handling rules are defined by statute and regulation and change over time; read the current rule and the partner's interpretation before accepting data. In the EU, CER and NIS2 impose confidentiality on information exchanged with authorities. Personal data in interview notes, rosters and call recordings fall under ordinary privacy law (for example, the GDPR in the EU).

This book does not give legal advice. It requires that the protocol (§1.8) name each regime that may apply, the person who decided whether it applies, and the handling consequence.

1.5.2 The information-sensitivity test

Even without a formal regime, apply a sensitivity test to every artifact: Does it identify a specific asset, its location, its vulnerability or its dependency in a way that would help someone harm it? Feeder-to-facility maps, generator fuel capacities, backup-communication paths and restoration priorities are exactly what an adversary planning a cascading effect wants. The Cascade Study's central data — which feeder serves which critical facility and how long its buffer lasts — is therefore sensitive even when every number is approximate. Chapter 7 builds the release tiers that follow from this.

1.5.3 Agreements before access

Negotiate before collecting anything: a data-use agreement (scope, storage, retention, destruction, breach notification); a publication agreement (the partner may review for protected information and factual error within a fixed period, but may not veto findings it dislikes); an authorship and acknowledgement policy; and a staff-protection clause (research data will not be used for discipline). Record the agreements in the protocol's appendix. A study without a publication agreement has, in practice, given its partner a veto.

1.6 Measurement: resilience and consequence metrics

1.6.1 The performance curve

Most quantitative resilience metrics summarize a performance curve Q(t): the fraction of normal service delivered over time, before, during and after a disruption. Bruneau et al. (2003) framed seismic community resilience in terms of reduced failure probabilities, reduced consequences and reduced recovery time, and their widely reproduced "resilience triangle" measures the loss as the area between full service and Q(t) over the recovery period. Haimes (2009) argued that resilience is a function of the specific threat and system state, not a single property. NIST's SP 800-160 Vol. 2 Rev. 1 (December 2021) frames cyber resiliency as the ability to anticipate, withstand, recover from and adapt to adverse conditions, stresses or attacks on systems that use or are enabled by cyber resources.

Common summaries of Q(t):

Metric Definition Sensitive to Blind to
Resilience loss (area) ∫ (1 − Q(t)) dt over the window Depth × duration Whether a short deep loss is worse than a long shallow one for people
Worst performance 1 − min Q(t) Depth Duration
Time to recovery First full-service time minus disruption start Duration Depth
Customer-hours lost Σ customers × hours without service Population and duration Criticality of the customers
Weighted critical-service loss Σ weight × hours a critical service is unavailable Predeclared priorities Ordinary customers, unless included

No metric is neutral. Each embeds a value judgment about whose loss counts and how depth trades against duration. A doctoral study must justify its primary metric by the decision it informs, report at least one alternative, and show whether conclusions change.

1.6.2 Censoring

If a service has not recovered when the observation window closes, its time to recovery is censored: the true value is at least the observed value. Averaging censored durations as if they were complete biases results toward faster recovery. Report the number censored, use survival methods when the sample allows, or extend the window. In simulations the window is a design choice; choose it so that censoring is rare under all policies compared, or report it.

1.6.3 Construct validity of consequence weights

A weighted consequence metric needs weights. Weights elicited from operators, emergency managers or published criticality frameworks are value judgments, not measurements. Predeclare them, cite where they came from, and run the analysis with alternative weight sets (Chapter 6). If the conclusion depends on one weight set, the claim must say so.

1.7 Common failure modes in infrastructure research claims

1.8 Writing a defensible research protocol

A protocol for an infrastructure study contains, at minimum:

  1. Question and claim rung. The research question, the intended claim, its rung (L1–L5) and its scope (systems, sectors, jurisdictions, hazards).
  2. Definitions and metrics. Each construct, its operationalization, primary and secondary metrics, window and censoring rule.
  3. Evidence sources. Which of the five sources (§1.2.1), why, and what each cannot vouch for.
  4. Design. Comparison conditions, baselines, scenario families, sampling, assignment.
  5. Pre-specification. What is fixed before results; the deviation log.
  6. Safety and ethics. Menlo analysis, hard rules (§1.4.2), physical-safety review, stop authority, human-subjects approval where applicable.
  7. Information protection. Applicable regimes, sensitivity test, storage, release tiers.
  8. Agreements. Data use, publication, authorship, staff protection.
  9. Analysis plan. Estimators, uncertainty, sensitivity analyses, alternative metrics and weights.
  10. Stop and abandonment conditions. When the study stops (safety, partner withdrawal, data breach) and when the claim is abandoned or narrowed (predeclared failure conditions, Chapter 3).
  11. Conflicts of interest and funding.
  12. Reproducibility plan. What will be released at which tier (Chapter 7).

1.9 Worked example: the Cascade Study's first protocol review

The doctoral candidate (synthetic) proposes to study restoration ordering in the Riverbend Region. The first draft says: "We will obtain REC's outage records and SCADA logs for the last five years, identify cyber-related outages and show that restoring critical customers first would have reduced harm." A protocol review finds:

Problem Why it fails Revision
"Cyber-related outages" in five years of one cooperative Likely zero or one event; no denominator; no variation to study. Use historical all-cause distribution outages to calibrate restoration durations, and a co-simulation for cyber scenarios.
"Would have reduced harm" Counterfactual with no model; hindsight. Explicit model with predeclared parameters; tabletop study with operators blind to the outcome.
"Critical customers first" Undefined; conflates customer count with criticality. Define critical services (telecom office, water treatment, hospital, dispatch), their buffers and predeclared weights.
SCADA logs requested High sensitivity; not needed for the question. Request outage-management summaries at feeder level, aggregated; no network or protocol data.
No partner agreement Partner veto by default. Data-use and publication agreements with a 30-day factual review and no veto on findings.
No safety metric A method that helps hospitals but leaves thousands of homes dark longer is a policy choice. Secondary metric: customer-hours lost, with a non-inferiority margin justified with the partner (Chapter 3).
Feeder–facility map would appear in the thesis Targeting value. Synthetic region in public text; real mapping only in controlled-access appendix, if at all.

The revised question becomes: In distribution outages that require manual, sequential feeder restoration, does ordering restoration by critical-service time-to-impact reduce weighted critical-service loss compared with ordering by customers restored, and by how much does it increase customer-hours lost? It is answerable, measurable, safe to study and honest about its trade-off.

1.10 Practical activity: resilience metrics can reverse rankings

The following offline lab computes three summaries of three synthetic performance curves and flags whether each is censored. Run it with Python 3; no packages are needed.

# Lab 1 - resilience metrics on three synthetic performance curves (offline, standard library only)
# Q(t) = delivered fraction of normal service (1.0 = full), sampled hourly over a 48-hour window.
def curve(drop_at, depth, hold, recover_hours, window=48):
    q = []
    for t in range(window + 1):
        if t < drop_at:
            q.append(1.0)
        elif t < drop_at + hold:
            q.append(1.0 - depth)
        elif t < drop_at + hold + recover_hours:
            frac = (t - drop_at - hold) / recover_hours
            q.append(round(1.0 - depth * (1 - frac), 4))
        else:
            q.append(1.0)
    return q

def metrics(q):
    loss = sum((1.0 - a) + (1.0 - b) for a, b in zip(q, q[1:])) / 2  # trapezoid, hour units
    worst = 1.0 - min(q)
    down = [t for t, v in enumerate(q) if v < 1.0]
    if not down:
        return round(loss, 2), worst, 0, False
    recovered = q[-1] >= 1.0
    ttr = (max(down) + 1 - min(down)) if recovered else None
    return round(loss, 2), round(worst, 2), ttr, not recovered

scenarios = {
    "A_deep_short": curve(drop_at=2, depth=0.90, hold=3, recover_hours=4),
    "B_shallow_long": curve(drop_at=2, depth=0.25, hold=20, recover_hours=10),
    "C_unrecovered": curve(drop_at=30, depth=0.40, hold=30, recover_hours=6),
}
print("scenario        loss_h  worst  ttr_h  censored")
for name, q in scenarios.items():
    loss, worst, ttr, cens = metrics(q)
    print(f"{name:15} {loss:6.2f}  {worst:5.2f}  {str(ttr):5}  {cens}")

Expected output:

scenario        loss_h  worst  ttr_h  censored
A_deep_short      4.95   0.90  7      False
B_shallow_long    6.38   0.25  30     False
C_unrecovered     7.40   0.40  None   True

Read the output carefully. By worst performance, A is far worse than B; by area loss and time to recovery, B is worse than A. Scenario C has the largest area loss in the window, but its loss is truncated at hour 48: the true loss is larger and its time to recovery is unknown. The lab demonstrates three points of §1.6: rankings depend on the metric; censoring hides the worst cases; and a single "resilience score" hides a value judgment. If A is a water-pressure loss that triggers a boil-water notice and B is a reduced-pressure period without health effect, neither area nor duration captures the decisive difference — a threshold on depth does.

1.11 Common errors and limits

The limit of this chapter is that integrity safeguards reduce, but do not remove, the risk of misleading results. Their purpose is to make errors detectable by others.

Exercises — chapter 1

EX001 — Claim-ladder audit. Objective: separate observations from consequence claims. Inputs: the synthetic statement "Logs show the attacker accessed REC's distribution management system at 01:52; the resulting outage left the hospital without power and patients were endangered; this shows that rural cooperatives are the weakest point in the grid." Constraints: use the five rungs of §1.1.3. Steps: (1) split the statement into its component assertions; (2) assign each a rung; (3) name the hidden assumption behind each; (4) name one observation that would defeat each; (5) rewrite the statement so that it claims only what the evidence could support. Expected evidence: table and rewrite. Rubric: 3 = every assertion correctly laddered, the hospital-generator buffer and the generalization flagged, rewrite properly bounded; 2 = one assertion mis-laddered; 1 = rewrite still overclaims; 0 = none.

EX002 — Definition and population. Objective: see how legal definitions change a study population. Inputs: NSM-22 (via the CISA page), the CER Directive and NIS2 as cited in §1.1.1. Constraints: do not paraphrase from memory; quote the definitional text you rely on, with article or section. Steps: (1) for a regional hospital, a small drinking-water utility and a payment processor, state whether each is in scope of each instrument and in which category; (2) name one consequence of the differences for a cross-jurisdiction comparative study. Expected evidence: 3 × 3 table with citations and a paragraph. Rubric: 3 = accurate with exact citations and a sound design consequence; 2 = one category error; 1 = uncited; 0 = none.

EX003 — Interdependency typing. Objective: apply Rinaldi et al.'s types. Inputs: the Riverbend dependencies: TEL on F2; HOSP on F2 and on WATER; WATER on F3; water SCADA on TEL; the electricity cooperative's billing on the payment processor; generator fuel deliveries on road access. Constraints: one primary type per relation, justified. Steps: classify each as physical, cyber, geographic or logical; identify any cycle; state which dependency a pure power-flow model would omit. Expected evidence: table and a paragraph. Rubric: 3 = types justified, absence of a cycle among the listed services noted, fuel/road and billing identified as outside a power-flow model; 2 = one misclassification; 1 = no justification; 0 = none.

EX004 — Menlo analysis. Objective: identify stakeholders and safeguards. Inputs: the revised Cascade Study question in §1.9, which includes interviews with REC dispatchers and a tabletop exercise. Constraints: use the four principles; at least six stakeholders. Steps: list stakeholders; for each principle, name the main risk and a safeguard; name the stop authority and the data that are destroyed if the partner withdraws. Expected evidence: stakeholder table and safeguard table. Rubric: 3 = served population, staff discipline risk, information sensitivity and partner pressure addressed with concrete safeguards; 2 = one principle thin; 1 = generic; 0 = none.

EX005 — Sensitivity test. Objective: decide what may be published. Inputs: five artifacts: (a) aggregated restoration durations by feeder class; (b) a list of hospitals with generator fuel days; (c) the model code; (d) the interview coding frame; (e) a map of which feeder serves the regional water treatment plant. Constraints: apply §1.5.2; propose a tier: public, controlled, withheld. Steps: (1) decide whether each artifact identifies a specific asset, location, vulnerability or dependency; (2) assign a tier; (3) justify the tier; (4) propose a transformation that would allow wider release. Expected evidence: table. Rubric: 3 = (b) and (e) withheld or transformed, (a) and (c) public, (d) public if no quotes, with transformations; 2 = one misjudged; 1 = all public or all withheld; 0 = none.

EX006 — Metric reversal. Objective: use Lab 1. Inputs: the lab code. Constraints: offline. Steps: (1) run the lab and record the output; (2) add a fourth metric, "hours below 0.5", and report it for all scenarios; (3) change the window to 72 hours and report what happens to C; (4) write 150 words on which metric you would choose as primary for a drinking-water service and why. Expected evidence: code change, outputs, paragraph. Rubric: 3 = correct values, C becomes uncensored with larger loss, metric choice justified by a health threshold; 2 = one value wrong; 1 = no justification; 0 = not run.

EX007 — Protocol skeleton. Objective: write an auditable protocol. Inputs: the revised question in §1.9. Constraints: all twelve elements of §1.8; at most three pages. Steps: (1) draft each of the twelve protocol elements of §1.8 in order; (2) mark each element as fixed or open; (3) write two stop conditions and two abandonment conditions; (4) list what a hostile reviewer would attack first. Expected evidence: protocol draft. Rubric: 3 = all elements concrete, stop and abandonment conditions distinct, information protection specific; 2 = one or two elements vague; 1 = missing safety or agreements; 0 = none.

Chapter synthesis

Infrastructure research studies consequences for people through chains of physical, cyber and organizational mechanisms that the researcher rarely observes directly. Its evidence comes from rare and selectively reported incidents, protected operational data, small testbeds, models and people; each source has characteristic biases and must be triangulated. Integrity safeguards — pre-specification, deviation logs, disclosure — are needed because partners are regulated, funders are interested and simulations are flexible. Safety and dual-use risks are real: never test on production, simulate first, separate capability from consequence in publication, and name a stop authority. Information about dependencies and buffers is sensitive even when approximate. Resilience metrics embed value judgments; choose them for the decision at hand and show whether conclusions survive alternatives. The protocol is the contract that makes all of this auditable.

Continue studying — chapter 01 of 08

← Course home · Course chapter index · Chapter 02 →

Continue to the next authored lesson.

Worked answers

EX001 — Claim-ladder audit.

# Assertion Rung Hidden assumption Defeating observation
1 Logs show access to the distribution management system (DMS) at 01:52. L1 if the log is preserved with clock state; otherwise L2 The log source is intact and its clock is correct; "attacker" is already an inference Log shows a scheduled vendor session; clock offset
2 The access was by an attacker. L2 The session was unauthorized Session matches an authorized maintenance ticket
3 The access caused the outage. L3 causal case claim Commands were issued and the feeder opened because of them, not protection Relay records show a protection trip; no command in DMS logs
4 The hospital lost power. L4 consequence The hospital's generator did not start or ran out Generator logs show seamless transfer
5 Patients were endangered. L4 consequence, strong Clinical services were affected Hospital reports no service change
6 Rural cooperatives are the weakest point in the grid. L5 research claim from one case One case represents a population; comparison with other utility types Any base-rate comparison

Rewrite: "In this synthetic case, the DMS recorded a session at 01:52 (DMS clock) whose authorization has not yet been established. A feeder serving the hospital opened at some point after; whether a command caused it, and whether the hospital lost power rather than transferring to its generator, has not been established. No conclusion about cooperatives in general follows from one case." Full credit requires flagging the generator buffer (the most likely reason the consequence claim is false) and the generalization.

EX002 — Definition and population. A defensible answer quotes: the statutory definition used by U.S. policy (42 U.S.C. § 5195c(e)) and NSM-22's sector structure via CISA's page; CER Article 2 definitions of "critical entity" and its annex of sectors; NIS2 Article 3 with Annexes I and II (essential and important entities). Typical classification: the regional hospital falls in the U.S. Healthcare and Public Health sector, is a likely NIS2 entity in the health sector depending on size and national rules, and may be designated a critical entity under CER only if the Member State identifies it; the small water utility is in the U.S. Water sector, may fall below NIS2 size thresholds unless a Member State applies the rules to it, and is in CER's drinking-water sector if designated; the payment processor is in the U.S. Financial Services sector, falls under financial-sector rules in the EU (NIS2 contains lex specialis provisions in relation to sector-specific acts such as DORA) and under CER's banking or financial-market infrastructure sectors if designated. Design consequence: a cross-jurisdiction comparison must define its population by a common functional criterion (for example, population served and function) rather than by legal category, and report the legal status of each unit as a covariate. The rubric rewards exact citations and treats designation-dependent cases as conditional rather than certain.

EX003 — Interdependency typing.

Relation Type Justification
TEL on F2 Physical Electricity is a material input
HOSP on F2 Physical Same
HOSP on WATER Physical Water is a material input
WATER on F3 Physical Same
Water SCADA on TEL Cyber Dependence on information carried by the telecom network
Billing on payment processor Logical (or cyber) Financial and contractual mechanism; affects operations only over days, not service delivery
Generator fuel on road access Physical, with a geographic component if roads and feeders share hazards Fuel is a material input; access may fail from the same local event

No cycle exists among the listed services (no service in the list depends back on HOSP, WATER or TEL). A pure power-flow model would omit the fuel–road dependency, the billing dependency and the TEL dependency of water SCADA. Full credit notes that the absence of a cycle is a property of this list, not of the region: telecom often needs water for cooling and power utilities need telecom for restoration, which would create cycles.

EX004 — Menlo analysis. Stakeholders: dispatchers interviewed; emergency managers in the tabletop; REC, RWA, hospital and carrier as organizations; the served population; regulators; the funder; the research team. Principle-by-principle: respect for persons — individual consent separate from employer permission, audio optional, withdrawal without penalty, no data to managers; beneficence — no contact with live systems, simulation only, sensitivity test on all artifacts, physical-safety review not required because nothing is energized; justice — do not burden only the smallest utility; report the customer-hour trade-off so no population's loss is hidden; law and public interest — data-use and publication agreements, applicable information-protection regimes identified, transparent methods. Stop authority: a named partner official and the principal investigator. On withdrawal: identifiable interview data destroyed; aggregated, de-identified contributions retained only if consent forms said so. The rubric rewards concrete safeguards, especially protection of staff from disciplinary use and treatment of the served population as a stakeholder.

EX005 — Sensitivity test.

Artifact Tier Reasoning and transformation
(a) Aggregated restoration durations by feeder class Public No specific asset or location; keep classes broad
(b) Hospitals with generator fuel days Withheld Identifies specific facilities and their buffer — targeting data. Transform to an anonymous distribution of fuel days across facility types if needed
(c) Model code Public Contains no infrastructure data; check that defaults are synthetic
(d) Interview coding frame Public No quotes or identifiers; quotes go to controlled tier
(e) Map of feeder serving the regional water plant Withheld Specific dependency of a specific asset. Describe in the thesis as "partner-supplied mapping, used for elicitation only"

EX006 — Metric reversal. Output with the added metric (hourly samples with Q < 0.5): A 5 h, B 0 h, C 0 h. With a 72-hour window, C's loss rises from 7.40 to 13.40 hour-units, its time to recovery becomes 36 h and it is no longer censored; A and B are unchanged. For drinking water, a defensible primary metric is hours below a pressure or quality threshold associated with a boil-water advisory or loss of fire-flow, because health consequence is threshold-driven; area loss and time to recovery are secondary. Full credit names the threshold's source (utility regulation or public-health guidance in the jurisdiction) rather than inventing a value, and notes that the 0.5 threshold in the lab is illustrative.

EX007 — Protocol skeleton. A strong protocol fixes the question and claim rung (L5 conditional, within a scenario class); defines weighted critical-service loss and customer-hours, a 72-hour window and censoring rule; names model, elicitation, tabletop and historical records as evidence sources with their limits; specifies the four policies and two regions; pre-specifies parameters, seeds, metrics, the non-inferiority margin and the decision rule; contains the Menlo analysis, the "never test on production" rule and human-subjects approval for interviews and the tabletop; names information-protection regimes and storage; attaches data-use and publication agreements; and lists stop conditions (partner request; any data breach; any participant distress) distinct from abandonment conditions (H2 fails in both regions; model fails historical validation by more than a predeclared margin). The hostile reviewer would attack weights, circularity and context selection first.