Chapter 42 — Critical infrastructure and sector security: applied research
Lesson map
Lesson 42 published. Map, example, EX083.
flowchart TD
N0["Learning outcomes and connection"]
N1["42.1 Engineering research versus academic research"]
N2["42.2 Designing a small study"]
N3["42.3 Public datasets and testbeds"]
N4["42.4 Ethics and authorisation"]
N5["42.6 Practical activity"]
N6["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX083"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["42.1 Engineering research versus academic research"]
N2["42.2 Designing a small study"]
N3["42.3 Public datasets and testbeds"]
N4["42.4 Ethics and authorisation"]
N5["42.6 Practical activity"]
N6["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX083"]
[Learning outcomes and connection]
|
v
[42.1 Engineering research versus academic research]
|
v
[42.2 Designing a small study]
|
v
[42.3 Public datasets and testbeds]
|
v
[42.4 Ethics and authorisation]
|
v
[42.6 Practical activity]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX083]
Learning outcomes and connection
After this chapter you can (1) frame an engineering question that needs new evidence; (2) design a small applied study — testbed experiment, simulation study or field measurement — with controls and reproducibility; (3) use public datasets and testbeds responsibly; and (4) report results with their limits. It is the engineering counterpart of the master's and doctoral research books and prepares the capstone and thesis (Chapters 44–49).
42.1 Engineering research versus academic research
This course's applied research answers a design question for a specific system, with evidence good enough for an engineering decision. It differs from the master's book, which teaches how to design and defend one ethically reviewed empirical study, and from the doctoral book, which asks for an original, independently validated contribution to knowledge. An engineering study succeeds if it changes or confirms a design decision with evidence that a reviewer can check.
42.2 Designing a small study
State the question and the decision it informs; define the measurement (Chapter 29); choose a design — comparison of alternatives on a testbed, a parameter sweep in simulation, or measurement of an existing system; fix the analysis before running; control for confounders (same load, same hardware, same seeds); repeat runs to measure variability; and record everything to allow reproduction (Sandve et al., 2013).
TMRP's detector study follows this pattern, with its analysis rule fixed before any run. Its seeds are recorded.
42.3 Public datasets and testbeds
Public industrial-security datasets and testbeds allow experiments without touching real infrastructure. The Secure Water Treatment (SWaT) testbed dataset described by Goh et al. (2017) and the BATADAL competition on water distribution networks reported by Taormina et al. (2018) are widely used. Know their limits: testbeds are smaller and cleaner than real plants, attack sets are chosen by researchers, and benchmark performance does not transfer automatically. Read evaluation pitfalls before claiming results (Arp et al., 2022; Sommer and Paxson, 2010).
42.4 Ethics and authorisation
Applied research on real systems requires the owner's written authorisation and, where people or personal data are involved, ethics review (Chapter 10). Results that reveal weaknesses in specific systems go to the owner, not to a conference.
For TMRP, the detector study uses simulation and LRWA's own historian data under the programme's authority. Any finding about a vendor's product would go to that vendor privately before anything is written for a wider audience, and no real station is used as an experimental subject.
42.5 Worked example: is the cumulative detector worth deploying? (synthetic)
Question: should TMRP deploy the cumulative mass-balance detector of Chapter 17 at Hollin Hill? Decision informed: deploy, with which threshold and recalibration interval. Study design: simulation sweep over noise levels (0.005–0.02 m), flow-meter drift (0–1%) and thresholds (0.02–0.08 m), 100 seeded runs per cell, measuring false alarms per month and detection time for ramp attacks of 0.01–0.1 m/h. Analysis fixed in advance: choose the threshold with fewer than one false alarm per month at the measured noise level and report detection time at that threshold. Validation: compare simulated noise with two weeks of historian data. Limits: perfect-mixing tank model; attacker who controls flow meters is out of scope.
42.6 Practical activity
Implement a reduced version of §42.5 using Lab 7: three thresholds × three noise levels × 20 seeds; report false alarms and ramp detection times in a table with variability.
Fix your analysis rule before running: for example, choose the lowest threshold with no false alarms in the no-attack runs at each noise level. Seed each run and record the seeds. Report the number of false alarms and the ramp detection time for every cell with its spread across seeds, not just the mean. Then state one limit of the reduced study, such as perfect flow meters, and the direction in which it biases your results.
42.7 Common errors and limits
- Studies without a decision they inform.
- Analysis chosen after seeing results.
- Single runs reported as results.
- Benchmark performance claimed to transfer to real plants.
- Weaknesses of real systems published without coordination.
The cumulative detector study in the worked example avoids each of these: it informs a deployment decision, fixes its analysis rule in advance, uses a hundred runs per cell, validates its noise model against two weeks of LRWA historian data and states that an attacker controlling the flow meters is out of scope.
Exercises — chapter 42
EX083 — Study protocol. Objective: write a pre-specified protocol. Inputs: §42.5. Steps: write a one-page protocol: question, decision, design, measurements, analysis rule, validation, limits. Expected evidence: protocol. Rubric: 3 = complete, analysis rule fixed in advance, validation against real noise data, limits honest; 2 = analysis vague; 1 = no decision; 0 = none.
EX084 — Dataset limits. Objective: assess transfer from a testbed. Inputs: a vendor claims 99% detection on a public water testbed dataset. Steps: list five questions you would ask before accepting this for LRWA. Expected evidence: list. Rubric: 3 = base rate and false-alarm rate in operations, attack set realism, evaluation protocol (point adjustment, data leakage), plant scale/difference, adaptation to adversaries; 2 = three good questions; 1 = generic; 0 = none.
Chapter synthesis
Engineering research answers a design question with checkable evidence. Pre-specify the study, control confounders, repeat runs, use public testbeds within their limits, obtain authorisation for anything real, and report results with their boundaries.
Continue studying — chapter 42 of 50
← Chapter 41 · Course chapter index · Chapter 43 →
Continue to the next authored lesson.