Chapter 4 — Advanced risk and ethics
Lesson map
Lesson 04 published. Map, example, EX016.
flowchart TD
N0["Learning outcomes and connection"]
N1["4.1 Why point estimates mislead"]
N2["4.2 Appetite and tolerance in service units"]
N3["4.3 Expert judgement as evidence"]
N4["4.4 Ethics in essential-service security"]
N5["4.5 Who bears residual risk"]
N6["4.7 Practical activity: loss exceedance and distribu"]
N7["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX016"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["4.1 Why point estimates mislead"]
N2["4.2 Appetite and tolerance in service units"]
N3["4.3 Expert judgement as evidence"]
N4["4.4 Ethics in essential-service security"]
N5["4.5 Who bears residual risk"]
N6["4.7 Practical activity: loss exceedance and distribu"]
N7["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX016"]
[Learning outcomes and connection]
|
v
[4.1 Why point estimates mislead]
|
v
[4.2 Appetite and tolerance in service units]
|
v
[4.3 Expert judgement as evidence]
|
v
[4.4 Ethics in essential-service security]
|
v
[4.5 Who bears residual risk]
|
v
[4.7 Practical activity: loss exceedance and distribu]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX016]
Learning outcomes and connection
After this chapter you should be able to:
- Build a quantified risk estimate with explicit frequency and severity uncertainty, and read it as a loss exceedance curve rather than a single number.
- Express risk appetite and tolerance in service units, with limits on the tail, not only the average.
- Elicit and document expert judgement so that it can be challenged.
- Analyse who bears residual risk, and recognize when an option that is better on average is worse for a specific group.
- Apply recognized ethical principles to security decisions, testing and disclosure in essential services.
Chapter 1 compared options under a table of scenarios. This chapter replaces the table with distributions and adds the question decision tables hide: for whom is an option better? Chapter 8 applies both to cases where every choice harms someone.
4.1 Why point estimates mislead
A risk register entry such as "likelihood 3, impact 4, score 12" hides three things a decision needs: how often, how bad, and how uncertain. NIST's risk-assessment guidance describes risk as a function of threat, vulnerability, likelihood and impact, and warns that assessments must state assumptions and uncertainty (NIST SP 800-30 Rev. 1). Multiplying ordinal scores also does arithmetic on numbers that are only labels.
A better minimal model has two parts: a frequency distribution for how many loss events occur in a year, and a severity distribution for how bad each is. Severity in infrastructure is usually right-skewed: most interruptions are short, a few are very long. A lognormal distribution is a common, simple choice for such quantities; its parameters should be anchored on evidence (past interruption durations, exercise results, peer incidents) and the choice itself recorded as an assumption. Simulating many years gives a distribution of annual loss, which you can summarize three ways: the mean (what you pay on average), the probability of exceeding thresholds that matter, and a high percentile such as the 99th (a one-in-a-hundred-year bad year). Plotting probability of exceedance against loss gives the loss exceedance curve (LEC).
4.2 Appetite and tolerance in service units
Risk appetite is the amount and type of risk an organization is willing to pursue or retain; tolerance sets the boundaries of acceptable variation. NIST CSF 2.0 places both under governance (NIST, 2024). For an essential service, appetite statements should be in service units and should constrain the tail. For example (synthetic): "Merrow Water will not accept a greater than 2% annual chance of a cyber-related interruption exceeding one million household-hours of supply, nor any credible scenario in which households in a single district lose supply for more than 48 hours without alternative provision." The first clause bounds the LEC; the second bounds distribution — a theme we return to in §4.5.
4.3 Expert judgement as evidence
Where data are thin, frequency and severity parameters come from people. Structured elicitation — defining quantities precisely, training experts on calibration, asking for quantiles rather than single values, aggregating several experts and recording rationale — makes judgement inspectable (O'Hagan et al., 2006). Practical rules for the specialist: ask about observable quantities ("how many hours to restore supply to Upper Merrow from a total SCADA loss?") rather than abstract probabilities; ask for a 90% range first, then a median; ask for base rates from comparable operators; and record the person, date, question and rationale so that the estimate can be revisited when evidence arrives.
4.4 Ethics in essential-service security
Security decisions in infrastructure distribute harm. Several established frameworks help reason about them:
- The Menlo Report adapted the principles of research ethics — respect for persons, beneficence, justice — to information and communication technology research and added respect for law and public interest (Dittrich and Kenneally, 2012). Its principles apply directly to testing and data collection in operational environments.
- Professional codes such as the ACM Code of Ethics and the NSPE Code of Ethics for Engineers place public welfare and safety above employer or client interest, and require honesty about the limits of one's competence and of a system's safety.
- Data-protection law imposes duties on processing personal data in security monitoring and incident handling (GDPR, Regulation (EU) 2016/679); privacy frameworks such as the NIST Privacy Framework help manage those duties in design.
For a specialist, four ethical questions recur. Distribution: who bears the residual risk of the chosen option, and did they have a voice? Proportionality: is the intrusion of a security measure — monitoring of staff, restriction of patients' or customers' access — proportionate to the harm it prevents? Honesty: are risks and uncertainties communicated truthfully to decision-makers and, where appropriate, to the public? Disclosure: when you discover a weakness that also affects others — a vendor product used by other utilities — how do you protect them without arming attackers? Coordinated vulnerability disclosure processes exist for exactly this case (CISA, CVD Program).
4.5 Who bears residual risk
Aggregate measures — total household-hours lost, expected loss — can favour options that protect dense, well-served areas, because protecting many people a little scores more than protecting few people a lot. A specialist should therefore report, alongside totals, per-capita burden by group (district, customer type) and burden on people least able to cope, such as households with medical dependence on water or power. Neither totals nor per-capita measures settle the question; the choice between them is a value judgement that belongs to the accountable decision-maker, informed by legal duties (for example, duties to vulnerable customers) and by the organization's stated appetite. The specialist's ethical duty is to make the distribution visible so the trade-off is chosen knowingly.
4.6 Worked example: two options with the same budget (synthetic)
Merrow Water serves three districts. Harbour is urban (60,000 households); Fenley suburban (25,000); Upper Merrow rural (6,000 households on a single trunk main, with telemetry that is slow to restore and long travel times for crews). Cyber-related supply interruptions are modelled as Poisson events with lognormal duration (median 8 hours). Upper Merrow's restoration currently takes three times as long. Two options cost the same:
- X: harden central SCADA — reduces event frequency from 0.30 to 0.20 a year everywhere; does nothing for Upper Merrow's long restoration.
- Y: second feed and local control for Upper Merrow — leaves frequency at 0.30, but cuts Upper Merrow restoration to 0.75 of the base duration.
Vulnerable households (registered medical or care needs) are counted separately: 900 in Harbour, 500 in Fenley, 400 in Upper Merrow.
4.7 Practical activity: loss exceedance and distribution
# Lab 4 - same budget, different people: loss exceedance and the distribution of harm (offline, stdlib only)
# Synthetic: Merrow Water supply interruptions caused by cyber events. Two options cost the same.
import random, math
random.seed(4)
YEARS = 20000
DISTRICTS = { # name: (households, share of interruption events that hit this district, vulnerable households)
"Harbour (urban)": (60000, 0.40, 900),
"Fenley (suburban)": (25000, 0.25, 500),
"Upper Merrow (rural, single main)": (6000, 0.35, 400),
}
def poisson(lam):
L, k, p = math.exp(-lam), 0, 1.0
while True:
p *= random.random()
if p < L: return k
k += 1
def simulate(freq, hours_mu, hours_sigma, rural_hours_factor):
annual_hh_hours = []; per_district = {d: 0.0 for d in DISTRICTS}; vuln = {d: 0.0 for d in DISTRICTS}
for _ in range(YEARS):
total = 0.0
for _ in range(poisson(freq)):
hours = random.lognormvariate(hours_mu, hours_sigma)
r = random.random(); acc = 0
for d, (hh, share, v) in DISTRICTS.items():
acc += share
if r < acc:
h = hours * (rural_hours_factor if d.startswith("Upper") else 1.0)
per_district[d] += hh * h; vuln[d] += v * h; total += hh * h
break
annual_hh_hours.append(total)
return annual_hh_hours, per_district, vuln
OPTIONS = {
"X: harden central SCADA (fewer events)": (0.20, math.log(8), 0.9, 3.0),
"Y: second feed + local control Upper Merrow": (0.30, math.log(8), 0.9, 0.75),
}
for name, args in OPTIONS.items():
loss, per, vul = simulate(*args)
loss.sort()
mean = sum(loss) / YEARS
p_any = sum(1 for x in loss if x > 0) / YEARS
p_big = sum(1 for x in loss if x > 1_000_000) / YEARS
p99 = loss[int(0.99 * YEARS)]
print(name)
print(f" mean household-hours/yr {mean:9.0f} P(any)={p_any:.3f} P(>1M hh-h)={p_big:.4f} 99th pct {p99:9.0f}")
for d, (hh, share, v) in DISTRICTS.items():
print(f" {d:34s} hours per household/yr {per[d]/YEARS/hh:5.2f} vulnerable-household-hours/yr {vul[d]/YEARS:7.0f}")
Expected output:
X: harden central SCADA (fewer events)
mean household-hours/yr 88346 P(any)=0.183 P(>1M hh-h)=0.0204 99th pct 1498603
Harbour (urban) hours per household/yr 0.97 vulnerable-household-hours/yr 872
Fenley (suburban) hours per household/yr 0.64 vulnerable-household-hours/yr 318
Upper Merrow (rural, single main) hours per household/yr 2.39 vulnerable-household-hours/yr 956
Y: second feed + local control Upper Merrow
mean household-hours/yr 117262 P(any)=0.261 P(>1M hh-h)=0.0305 99th pct 1920694
Harbour (urban) hours per household/yr 1.48 vulnerable-household-hours/yr 1332
Fenley (suburban) hours per household/yr 0.91 vulnerable-household-hours/yr 453
Upper Merrow (rural, single main) hours per household/yr 0.96 vulnerable-household-hours/yr 386
Interpretation. On every aggregate measure X is better: mean loss 88,346 against 117,262 household-hours a year; a 2.0% against 3.1% annual chance of exceeding one million household-hours; a lower 99th percentile. Against the appetite in §4.2, X sits on the 2% tail bound (0.0204, within simulation error of the limit) and Y clearly breaches it. But the distribution reverses the picture. Under X, an Upper Merrow household loses on average 2.39 hours a year — more than twice an urban household and nearly four times a suburban one — and Upper Merrow's 400 vulnerable households bear 956 vulnerable-household-hours a year, more than Harbour's 900 households. Y cuts the rural burden to 0.96 hours per household and rural vulnerable-household-hours to 386, at the cost of higher burdens elsewhere. Total vulnerable-household-hours are almost identical (about 2,146 under X and 2,171 under Y): the options do not differ much in how much harm falls on vulnerable people overall, but sharply in which vulnerable people bear it. The second appetite clause — no district without supply beyond 48 hours — would require looking at Upper Merrow's tail, which the summary does not show; that is EX016's extension.
The analysis does not decide between X and Y. It shows that "X is better" is true only under an aggregate-utilitarian view and that the choice concentrates risk on a small rural population. A defensible recommendation might combine X with cheaper non-technical measures for Upper Merrow (pre-positioned bottled water, a priority register for vulnerable households), and must record the value judgement.
4.8 Common errors and limits
- Multiplying ordinal scores and treating the product as a quantity.
- Reporting the mean annual loss alone for a heavy-tailed risk.
- Stating risk appetite as a colour or adjective rather than a bound in service units.
- Treating elicited numbers as data while discarding the elicitation record.
- Reporting only aggregate benefit, so that concentrated harm on small groups is invisible.
- Presenting a value judgement (aggregate versus distribution) as a technical result.
- Sitting on a vulnerability that affects other operators because disclosure is awkward.
The model's limits are those of its assumptions: Poisson independence ignores campaigns that hit several sites at once; the lognormal may understate the longest interruptions; district shares are judgements. Change them and the numbers move; the method of showing distribution alongside totals does not.
Exercises — chapter 4
EX016 — Partial rural improvement. Objective: see how distribution responds to a weaker option. Inputs: Lab 4. Constraints: change Y's rural duration factor from 0.75 to 1.5 (the second feed is delayed; only local control is delivered). Steps: (1) rerun; (2) report Upper Merrow's hours per household and vulnerable-household-hours under the modified Y; (3) compare with X and the original Y; (4) extend the lab to report the probability that Upper Merrow has a single interruption longer than 48 hours in a year under X and under Y. Expected evidence: outputs and paragraph. Rubric: 3 = modified-Y numbers correct (1.93 hours; 771 vulnerable-household-hours), still better for Upper Merrow than X but much less than original Y, 48-hour extension implemented with a correct per-event check; 2 = extension missing; 1 = not run; 0 = none.
EX017 — Ethical analysis memo. Objective: apply ethical principles to the X/Y choice. Inputs: §4.4–4.7. Constraints: 400 words; principles from the Menlo Report and one professional code. Steps: (1) identify affected groups and their voice; (2) analyse beneficence and justice; (3) state the value judgement the board must make; (4) recommend a combined option with non-technical measures. Expected evidence: memo. Rubric: 3 = groups specific, trade-off between aggregate and distributional views named as a judgement, combined recommendation concrete, duty of honesty to the board observed; 2 = principles listed but not applied; 1 = asserts X or Y as technically correct; 0 = none.
EX018 — Elicitation protocol. Objective: obtain defensible severity parameters. Inputs: the question "how long would it take to restore supply to Upper Merrow after total loss of SCADA?" Constraints: three experts; observable quantity; quantiles. Steps: (1) define the quantity precisely; (2) write the elicitation script including a calibration question; (3) explain how you would aggregate and record. Expected evidence: protocol. Rubric: 3 = quantity unambiguous (start and end events defined), 5th/50th/95th quantiles asked, independent elicitation before discussion, calibration and record kept; 2 = single values asked; 1 = probability of a vague event; 0 = none.
EX019 — Appetite statement. Objective: write appetite and tolerance in service units. Inputs: Merrow Health (three hospitals, an ambulance service dependency). Constraints: at least one tail bound and one distribution bound; measurable from the organization's own data. Steps: draft; then test it against one plausible scenario and say whether it is breached. Expected evidence: statement and test. Rubric: 3 = service units (for example hours of emergency department diversion, delayed time-critical treatments), tail and distribution bounds measurable, test honest; 2 = only averages; 1 = adjectives; 0 = none.
EX020 — Disclosure dilemma. Objective: handle a weakness that affects others. Inputs: during a review, Merrow Water finds that Corvane's remote-support tool accepts a default credential on a maintenance port; Corvane supports other utilities in the region. Constraints: no testing beyond Merrow's own systems; no public disclosure before a coordinated process. Steps: (1) immediate protective actions for Merrow; (2) notification route to Corvane and, if needed, a coordinator; (3) what to tell the resilience forum, and when; (4) ethical justification. Expected evidence: plan. Rubric: 3 = protect own systems first, private notification to vendor with evidence, use of a coordinated disclosure process if the vendor does not act, forum informed in a way that protects others without publishing exploit details, principles cited; 2 = one step missing; 1 = tests other utilities' systems or publishes; 0 = none.
Chapter synthesis
Quantify risk as frequency and severity with explicit uncertainty, and read it through the loss exceedance curve, not a single score. Write appetite and tolerance in service units with bounds on the tail and on distribution. Treat expert judgement as evidence that must be elicited carefully and recorded. Show who bears residual risk — per capita and for those least able to cope — because options that are better on average can concentrate harm. Apply recognized ethical principles to choices, testing and disclosure, and keep value judgements visible as judgements for the accountable decision-maker.
Continue studying — chapter 04 of 12
← Chapter 03 · Course chapter index · Chapter 05 →
Continue to the next authored lesson.