Chapter 8 — Critical infrastructure and sector security: failure modes
Lesson map
Lesson 08 published. Map, example, EX029.
flowchart TD
N0["Learning outcomes and connection"]
N1["8.1 A catalogue of failure modes"]
N2["8.2 Rules that explain themselves"]
N3["8.3 In-range is not true"]
N4["8.5 Choosing thresholds"]
N5["8.6 Practical activity"]
N6["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX029"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["8.1 A catalogue of failure modes"]
N2["8.2 Rules that explain themselves"]
N3["8.3 In-range is not true"]
N4["8.5 Choosing thresholds"]
N5["8.6 Practical activity"]
N6["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX029"]
[Learning outcomes and connection]
|
v
[8.1 A catalogue of failure modes]
|
v
[8.2 Rules that explain themselves]
|
v
[8.3 In-range is not true]
|
v
[8.5 Choosing thresholds]
|
v
[8.6 Practical activity]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX029]
Learning outcomes and connection
After this chapter you should be able to:
- name the common technical failure modes of telemetry and control — gap, stuck value, spike, drift, out-of-range, stale timestamp, loss of control authority — and their usual causes;
- detect them in a time series with simple, explainable rules;
- explain why an in-range value can still be false, and why a perfectly constant value is suspicious;
- keep fault and attack hypotheses open together until evidence separates them.
Chapter 7 put events in order. This chapter asks of each measurement: is it true? Many security events first appear as "bad data", and many bad-data events are ordinary faults. A technician must recognize both.
8.1 A catalogue of failure modes
| Mode | Looks like | Ordinary causes | Security-relevant causes |
|---|---|---|---|
| Gap | Missing samples | Link outage, polling overload (Chapter 1), device reboot | Blocked or dropped traffic, denial of service |
| Stuck / flatline | Identical values for longer than the process allows | Frozen sensor, failed analog input, SCADA serving last known value | Replayed or frozen values hiding a real change |
| Spike | A jump faster than physics permits | Electrical noise, scaling error, bad sample | Injected value |
| Drift | Slow divergence from reality | Sensor fouling, calibration age | Slow manipulation (rare, hard to detect) |
| Out of range | Impossible value | Wiring fault, wrong scaling | Crafted value |
| Stale timestamp | Value with an old or future time | Clock error (Chapter 7), buffered data | Replay |
| Loss of control authority | Commands not executed or executed by someone else | Local/remote switch left in local, interlock | Unauthorized write |
The point of the table is not that every gap is an attack. It is that the same symptom has both kinds of cause, so detections should report the symptom precisely, and investigation (Chapter 13) separates the causes.
8.2 Rules that explain themselves
Each detection in this chapter is a rule a person can read and verify:
- Gap: more than N consecutive missing samples, where N is set from the poll cycle and the process's tolerance.
- Stuck: more than M identical consecutive readings, where M reflects how much a healthy sensor naturally varies. A chlorine analyser reporting exactly 0.62 for twenty minutes is suspicious because real analysers fluctuate in their last digit.
- Spike: a change between consecutive samples larger than the maximum physically plausible step — a 12 m tank cannot rise 6.8 m in a minute. This catches values that range checks miss.
- Range: outside the instrument or process limits (Chapter 4).
These rules are crude but transparent: when one fires, everyone can see why. Statistical or learned anomaly detectors can add sensitivity later; they should not replace rules that operators understand.
8.3 In-range is not true
A value of 9.80 m on a 12 m tank passes every range check. If the previous minute read 2.96 m and the next 2.97 m, the 9.80 is impossible as a measurement. Conversely, a constant value is always "in range". Range checks protect against the absurd; rate and variation checks protect against the plausible-looking false.
Use a maximum step from the process, such as half a metre per minute for a tank level, so that a single wild sample is caught even when it lies inside the instrument's range. Write the step limit next to the range limit in the tag's schema, and review both after every false alarm.
8.4 Worked example: four tags, twenty minutes (synthetic)
Lab 8 (save as lab08.py, run with python3 lab08.py):
# Lab 8 - classify telemetry failure modes (offline, standard library only)
# Synthetic one-minute readings from four Kestrel Water tags, 06:00-06:19Z on 2026-10-12.
series = {
"KW-01 LEVEL_M": [3.40,3.41,3.41,3.42,3.43,3.43,3.44,3.45,3.45,3.46,3.47,3.47,3.48,3.49,3.49,3.50,3.51,3.51,3.52,3.53],
"KW-02 CL2_MGL": [0.62,0.62,0.62,0.62,0.62,0.62,0.62,0.62,0.62,0.62,0.62,0.62,0.62,0.62,0.62,0.62,0.62,0.62,0.62,0.62],
"KW-05 FLOW_LS": [41.0,40.8,41.2,40.9,None,None,None,None,None,None,None,41.1,40.7,41.0,41.3,40.9,41.2,41.0,40.8,41.1],
"KW-07 LEVEL_M": [2.95,2.96,2.95,2.97,2.96,2.96,9.80,2.97,2.96,2.98,2.97,2.97,2.98,2.97,2.99,2.98,2.98,2.99,2.98,2.99],
}
LIMITS = {"LEVEL_M": (0.0, 12.0), "CL2_MGL": (0.0, 5.0), "FLOW_LS": (0.0, 400.0)}
MAX_STEP = {"LEVEL_M": 0.5, "CL2_MGL": 0.3, "FLOW_LS": 25.0} # physically plausible change per minute
STUCK_MIN, GAP_MIN = 15, 5
for name, xs in series.items():
tag = name.split()[1]; lo, hi = LIMITS[tag]; out = []
run, longest = 0, 0
for i in range(len(xs)): # gaps
run = run + 1 if xs[i] is None else 0; longest = max(longest, run)
if longest >= GAP_MIN: out.append(f"gap: {longest} consecutive missing minutes")
vals = [(i, v) for i, v in enumerate(xs) if v is not None]
same, best = 1, 1
for (i0, a), (i1, b) in zip(vals, vals[1:]): # stuck / flatline (exact repeats)
same = same + 1 if a == b else 1; best = max(best, same)
if abs(b - a) > MAX_STEP[tag] and i1 - i0 == 1:
out.append(f"spike: 06:{i1:02d}Z jump {a} -> {b} exceeds {MAX_STEP[tag]}/min")
if best >= STUCK_MIN: out.append(f"stuck: {best} identical readings in a row")
out += [f"range: {v} outside {lo}-{hi}" for _, v in vals if not lo <= v <= hi]
print(f"{name:<14} {'; '.join(out) if out else 'no failure mode detected'}")
Expected output (Python 3.13.5):
KW-01 LEVEL_M no failure mode detected
KW-02 CL2_MGL stuck: 20 identical readings in a row
KW-05 FLOW_LS gap: 7 consecutive missing minutes
KW-07 LEVEL_M spike: 06:06Z jump 2.96 -> 9.8 exceeds 0.5/min; spike: 06:07Z jump 9.8 -> 2.97 exceeds 0.5/min
KW-01 behaves like a slowly filling tank — no finding. KW-02's chlorine is identical for all twenty readings: either the analyser has frozen, the RTU is returning a stored value, or the value is being supplied by something other than the analyser. KW-05 has a seven-minute gap — check the link and polling first (Chapter 13). KW-07 shows a single-sample excursion to 9.80 m and back: a classic noise or scaling glitch, but also exactly what a one-off injected write to a reporting register would look like. Each line is a symptom with a precise description; none is yet a cause.
8.5 Choosing thresholds
Set thresholds from evidence, not guesses: look at a week of healthy data per tag, find the largest natural step and the longest natural run of identical values, and set the rule above them with a margin. Record the reason beside the number, and review after every false alarm and every missed event. A stuck-value rule on a tag that legitimately sits at 0.00 (a pump that is off) needs a condition — for example, apply it only while the pump runs.
8.6 Practical activity
Add a drift check to Lab 8: compare each tag's mean over the first five minutes with its mean over the last five minutes and flag a change larger than a tag-specific limit. Discuss why KW-01's steady rise should not be flagged, and what extra information (pump state, inflow) would make drift detection meaningful.
KW-01's steady rise is process behaviour while a pump fills the tank, so a naive mean comparison would flag it wrongly. Drift becomes meaningful when you know the pump state or inflow, or when you compare two sensors that should agree. Set the tag-specific limit from a week of healthy data, and record the reason beside the number.
8.7 Common errors and limits
- Treating every bad value as an attack — or none.
- Range checks only.
- Stuck-value rules on tags that legitimately sit still.
- Thresholds with no recorded rationale.
-
Lab 8 sees one series at a time; real investigation compares related tags (a pump running while flow is zero is a cross-tag inconsistency).
-
Applying a stuck-value rule to a tag that legitimately sits at zero while a pump is off.
- Setting thresholds from a guess instead of from healthy data with a recorded rationale.
- Investigating one series in isolation when a pump-running bit and a zero flow would show a cross-tag fault.
Exercises — chapter 8
EX029 — Name the mode. Objective: classify symptoms. Inputs: (a) level rises 0.01 m/min while the inlet valve reports closed; (b) flow reads 0.00 for 40 minutes while pump 1 reports running; (c) the historian shows values timestamped 03:00–03:10 that arrived at 03:25; (d) pump 2 does not start after an HMI command and the RTU reports "local". Steps: name the mode and two candidate causes each. Expected evidence: four rows. Rubric: 3 = (a) cross-tag inconsistency/drift — valve feedback fault or a leak/bypass; (b) stuck or cross-tag inconsistency — failed flow meter or frozen value; (c) buffered/stale data — link outage with store-and-forward, or replay; (d) loss of control authority — local switch at site, or interlock; 2 = modes right, causes thin; 1 = partial; 0 = none.
EX030 — Predict a rule change. Objective: reason about thresholds. Inputs: Lab 8 with STUCK_MIN = 25 and MAX_STEP["LEVEL_M"] = 7.0. Steps: predict the new output, then run. Expected evidence: prediction and actual output. Rubric: 3 = KW-02 no longer flagged (20 < 25), KW-07 spike no longer flagged (6.84 < 7.0, 6.83 < 7.0), KW-05 gap still flagged; explains what is lost; 2 = mostly right; 1 = guesses; 0 = none.
EX031 — Conditional stuck rule. Objective: reduce false alarms. Inputs: a flow tag legitimately reads 0.0 when its pump is off. Steps: write the rule in pseudocode using pump state. Expected evidence: rule and a test case. Rubric: 3 = apply stuck detection only while pump state = running for at least the poll interval; includes a test with pump off (no alert) and running with constant non-zero flow (alert); 2 = no test; 1 = disables the rule; 0 = none.
EX032 — Fault or attack plan. Objective: keep both hypotheses open. Inputs: KW-02 stuck at 0.62. Steps: list three checks that would point to a fault and three that would point to manipulation, all read-only. Expected evidence: six checks. Rubric: 3 = fault: analyser local display, RTU analog input diagnostics, maintenance records; manipulation: unexpected writes to the reporting register in captures, configuration session logs, comparison with a manual sample by operations; 2 = one side thin; 1 = one hypothesis only; 0 = none.
Chapter synthesis
Telemetry fails in a small number of recognizable ways, and each symptom has both ordinary and malicious causes. Detect symptoms with transparent rules — gap, stuck, spike, range — whose thresholds come from healthy data and whose reasons are written down. In-range is not the same as true. Report the symptom precisely and leave the cause to investigation.
Continue studying — chapter 08 of 18
← Chapter 07 · Course chapter index · Chapter 09 →
Continue to the next authored lesson.