Chapter 32 — Critical infrastructure and sector security: reliability
Lesson map
Lesson 32 published. Map, example, EX063.
flowchart TD
N0["Learning outcomes and connection"]
N1["32.1 Measures"]
N2["32.2 Repairable redundancy"]
N3["32.3 Induced failures"]
N4["32.5 Practical activity: Lab 12"]
N5["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX063"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["32.1 Measures"]
N2["32.2 Repairable redundancy"]
N3["32.3 Induced failures"]
N4["32.5 Practical activity: Lab 12"]
N5["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX063"]
[Learning outcomes and connection]
|
v
[32.1 Measures]
|
v
[32.2 Repairable redundancy]
|
v
[32.3 Induced failures]
|
v
[32.5 Practical activity: Lab 12]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX063]
Learning outcomes and connection
After this chapter you can (1) use reliability measures — failure rate, MTBF, MTTR, availability; (2) compute availability of series, parallel and repairable redundant systems with Markov models; (3) show how repair capacity and common causes change redundancy benefits; and (4) connect reliability engineering to security, where failures may be induced. It builds on Chapter 7 and prepares Chapters 33 and 41.
32.1 Measures
For a component with constant failure rate $\lambda$ (failures per hour), mean time between failures is $1/\lambda$. With mean time to repair $1/\mu$, steady-state availability is $A = \mu/(\lambda+\mu) = \text{MTBF}/(\text{MTBF}+\text{MTTR})$. Constant failure rates suit the useful-life period of electronics; early-life and wear-out failures need other distributions (Weibull). Components in series (all needed) have rates that add: $\lambda_s = \sum\lambda_i$. Components in parallel (any one suffices) fail only together.
At PS4, each pump's failure rate of one per two thousand hours and its 24-hour repair time give the single-pump availability used in Lab 12.
32.2 Repairable redundancy
Simple formulas for parallel systems assume independent units each repaired independently. Real sites often have one repair crew, so when both units are down one waits. A continuous-time Markov chain models this: states 0, 1, 2 units failed; transitions at rate $2\lambda$ from 0 to 1, $\lambda$ from 1 to 2, and $\mu$ back down from 1 and from 2 with one crew. Solving the balance equations gives steady-state probabilities; unavailability is the probability of state 2.
PS4 has exactly this constraint: two pumps, one repair crew. The Markov model in Lab 12 captures the wait.
32.3 Induced failures
Reliability models assume failures arrive randomly. An adversary chooses when. If an attacker can disable one pump remotely and wait for the other to fail — or disable both — the Markov numbers no longer describe the risk. Reliability engineering still matters for security: it tells you the baseline the attacker must overcome, the repair capacity that determines recovery speed, and the spare holdings that make cold standby real. Security requirements then aim to make induced failures require independent compromises (Chapter 23's conditioning).
32.4 Worked example: PS4 duty/standby pumps (synthetic)
Each PS4 pump has a failure rate of 1 per 2,000 h and mean repair time of 24 h with one crew. Lab 12 gives single-pump availability 0.98814 (about 104 h down a year). The 1-out-of-2 pair with one crew has availability 0.999719 — both pumps down about 148 minutes a year. The naive formula $1-(1-A)^2$ gives 0.999859, half the real unavailability, because it ignores that the second failure can occur while the single crew is busy. For the pump set's control chain — pump, starter and RTU in series with MTBFs of 2,000, 5,000 and 10,000 h — the series MTBF is 1,250 h: the chain is less reliable than its weakest element.
32.5 Practical activity: Lab 12
# Lab 12 - availability of a duplicated pump set with repair (two-state units, one repair crew; offline)
lam, mu = 1 / 2000, 1 / 24 # failures per hour per pump, repairs per hour
A1 = mu / (lam + mu)
# states: 0 = both up, 1 = one down, 2 = both down; one crew repairs one pump at a time
# balance: p1 = (2 lam / mu) p0 ; p2 = (lam / mu) p1
p0 = 1 / (1 + 2 * lam / mu + 2 * (lam / mu) ** 2)
p1 = 2 * lam / mu * p0; p2 = lam / mu * p1
print(f"single pump availability {A1:.6f} downtime h/yr {8760*(1-A1):.1f}")
print(f"1oo2 availability {1-p2:.8f} P(both down) {p2:.2e} downtime min/yr {8760*60*p2:.2f}")
print(f"naive independent 1-(1-A1)^2 = {1-(1-A1)**2:.8f}")
mtbf_series = 1 / (1/2000 + 1/5000 + 1/10000)
print(f"series MTBF pump+starter+RTU: {mtbf_series:.0f} h")
Expected output:
single pump availability 0.988142 downtime h/yr 103.9
1oo2 availability 0.99971883 P(both down) 2.81e-04 downtime min/yr 147.78
naive independent 1-(1-A1)^2 = 0.99985939
series MTBF pump+starter+RTU: 1250 h
The single pump is down about 104 hours a year. The pair with one crew is down together for about 148 minutes a year, roughly twice what the naive formula suggests, because the second failure can arrive while the crew is still repairing the first. The series figure shows the control chain's weakness: pump, starter and RTU together have a mean time between failures of 1,250 hours, shorter than any one of them alone. Repair capacity, not pump quality, drives the difference.
32.6 Common errors and limits
- Independent-repair formulas with one crew.
- Constant failure rates for ageing equipment.
- Ignoring common causes (shared starter panel, shared power).
- Reliability numbers presented as protection against deliberate attack.
- Vendor MTBF figures used without knowing their test conditions.
For PS4 the first error halves the predicted downtime, and the third ignores that both pumps share one starter panel and one feeder. Vendor MTBF figures for the RTU should be checked against LRWA's own maintenance records before they enter the dossier, and none of these numbers says anything about an attacker who stops both pumps deliberately.
Exercises — chapter 32
EX063 — Two crews. Objective: model repair capacity. Inputs: Lab 12. Steps: modify the Markov model for two independent crews (repair rate $2\mu$ from state 2) and compute unavailability; compare with one crew and with the naive formula. Expected evidence: numbers and conclusion. Rubric: 3 = two-crew result matches the naive independent formula (explain why), one-crew roughly double; 2 = numbers right, no explanation; 1 = not run; 0 = none.
EX064 — Spares decision. Objective: connect reliability to logistics. Inputs: repair time 24 h assumes a spare pump in stock; without a spare, repair takes 21 days (504 h). Steps: compute 1oo2 unavailability with one crew and MTTR 504 h, and recommend whether to hold a spare. Expected evidence: number and recommendation. Rubric: 3 = correct computation (use the lab formula with μ = 1/504), recommendation tied to hospital consequence; 2 = number right; 1 = wrong rate; 0 = none.
Chapter synthesis
Reliability is measured with failure rates, repair times and availability; series chains are weaker than their weakest link; repairable redundancy depends on repair capacity, which simple formulas ignore. Adversaries induce failures on their schedule, so reliability gives the baseline and recovery capacity, while security must make induced failures require independent compromises.
Continue studying — chapter 32 of 50
← Chapter 31 · Course chapter index · Chapter 33 →
Continue to the next authored lesson.