OpenCyber AcademyLesson map
Lesson 09 published. Map, example, EX041.
flowchart TD
N0["Learning outcomes and connection"]
N1["9.1 Leading a specialist function"]
N2["9.2 Board reporting"]
N3["9.3 Signal or noise"]
N4["9.4 Assurance cases"]
N5["9.5 Evidence freshness"]
N6["9.6 Three lines and independent review"]
N7["9.8 Practical activity: XmR chart and evidence fresh"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX041"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["9.1 Leading a specialist function"]
N2["9.2 Board reporting"]
N3["9.3 Signal or noise"]
N4["9.4 Assurance cases"]
N5["9.5 Evidence freshness"]
N6["9.6 Three lines and independent review"]
N7["9.8 Practical activity: XmR chart and evidence fresh"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX041"]
[Learning outcomes and connection]
|
v
[9.1 Leading a specialist function]
|
v
[9.2 Board reporting]
|
v
[9.3 Signal or noise]
|
v
[9.4 Assurance cases]
|
v
[9.5 Evidence freshness]
|
v
[9.6 Three lines and independent review]
|
v
[9.8 Practical activity: XmR chart and evidence fresh]
|
v
[Worked example] --> [Exercise EX041]
Learning outcomes and connection
After this chapter you should be able to:
- Lead a security function in an essential service: set direction, build capability, and keep the trust of operations and the board.
- Report to a board in service units with a small set of indicators that inform decisions.
- Tell real change from noise in a security indicator using a process-behaviour (XmR) chart.
- Build an assurance case for a resilience claim and maintain the freshness of its evidence.
- Use the three lines model and independent review without turning assurance into paperwork.
Chapters 7 and 8 integrated security into the organization and tested it in difficult cases. This chapter is about the person responsible for it over years: what they report, how they know whether things are improving, and how they show others that claims are true. Chapter 10 then turns to the external evidence a leader must read.
9.1 Leading a specialist function
A security leader in an essential service leads people who are scarce, often stretched, and working across cultures — control engineers, IT staff, operators, suppliers. Four responsibilities dominate. Direction: a small number of priorities tied to the service and its consequence, revisited when evidence changes (Chapter 1's review triggers). Capability: the skills, tools and time to do the work, including succession, so that knowledge of a critical system does not live in one head. Trust: operations must believe that security understands the plant and will not endanger it; the board must believe reports are candid. Accountability: clear ownership (Chapter 7) and a record of decisions (Chapter 1). Professional codes require honesty about limits (ACM Code of Ethics); for a leader this includes telling the board what is not known.
Building capability in OT security is partly a matter of mixing backgrounds: control engineers who learn security and security specialists who learn the process. Rotations, joint exercises (Chapter 6) and shared incident reviews (Chapter 3) do more than courses alone.
9.2 Board reporting
Boards need to make decisions about appetite, investment and accountability. A good report is short and answers four questions: Are we within appetite? (Chapter 4's tail and distribution bounds, in service units); What changed? (new threats, incidents, assumptions); Are our controls working? (verified evidence, Chapter 2; evaluated performance, Chapter 6); What do you need to decide? Indicators should be few, defined, and tied to consequence: verified coverage of high-weight practices; time to remediate known-exploited vulnerabilities on exposed systems, using a reference list such as CISA's catalogue (CISA, KEV); last successful manual-operation drill per major site; recovery time demonstrated for critical systems; and open decisions with owners. Avoid counts that measure activity rather than outcome — numbers of alerts, trainings delivered, findings closed.
9.3 Signal or noise
Every indicator varies month to month. Reacting to every rise wastes attention; ignoring a real shift wastes time. A process-behaviour chart (also called an individuals and moving-range, or XmR, chart) separates the two using the indicator's own history. From a baseline period, compute the mean and the average absolute difference between consecutive values (the mean moving range); natural process limits are the mean plus or minus 2.66 times the mean moving range. Points outside the limits, or long runs on one side of the mean, signal a change worth investigating; points inside are routine variation (NIST/SEMATECH e-Handbook, control charts). Two practical rules: choose the baseline from a period believed stable and do not include the period you are testing; and treat a signal as a prompt to investigate, not as proof of a cause.
9.4 Assurance cases
An assurance case is a structured argument that a claim is true, supported by evidence. It follows the general structure of argument — claim, grounds, warrant, rebuttal (Toulmin, 2003) — applied to systems. For example: Claim: "Hollin can supply safe water for 72 hours without its control system." Sub-claims: manual dosing can be maintained by the staff on any shift; chemical stock lasts 72 hours; analysers work without SCADA; staff can be relieved. Evidence: drill reports with times and participants; stock records; analyser tests; rota. Rebuttals considered: night shift with minimum staff; analyser power loss; simultaneous incident at Fenley. An assurance case makes visible which evidence each claim depends on, so that when evidence ages or fails the affected claims are known.
9.5 Evidence freshness
Evidence decays. A restore tested eighteen months ago says little about today's backups; a firewall review before a network change says little after it. For each assurance claim, set a maximum evidence age related to how fast the underlying system changes, and report stale evidence as a gap, even if nothing is known to be wrong. A low-grade evidence type (an attestation) with a long allowed age deserves particular scrutiny: it may be "fresh" by the rule but weak by nature (Chapter 2's grades).
9.6 Three lines and independent review
The Institute of Internal Auditors' Three Lines Model describes roles for management (first line: owning and managing risk), specialist risk and compliance functions (second line: expertise, support and challenge) and internal audit (third line: independent assurance), all accountable to a governing body (IIA, 2020). For essential-service security, the model's value is clarity: operations owns the risk of its plant; the security function sets standards, challenges and helps; internal audit or an external reviewer tests that the system works. The model fails when the second line does the first line's work (so nobody in operations owns it) or when audit tests documents rather than reality.
9.7 Worked example: Merrow Water's board pack (synthetic)
Merrow Water's board receives monthly the median days to remediate known-exploited vulnerabilities on internet-facing systems. After a December reading of 24 days, a non-executive director asked whether things were getting worse or whether December was a bad month. At the same meeting, the specialist presented the evidence behind six assurance claims. Lab 9 examines both.
The director's question is the right one, and it cannot be answered by comparing December with November alone. The specialist needs a baseline from months when nothing unusual was known to have happened, so the chart uses the first eleven months and keeps the later readings out of it. The six claims were chosen because each supports the board's statement that Merrow Water can recover safely from a cyber incident; each has a named evidence type, a date and a maximum acceptable age agreed with internal audit.
9.8 Practical activity: XmR chart and evidence freshness
# Lab 9 - is the assurance signal a change or noise? An XmR chart and evidence freshness (offline, stdlib only)
# Synthetic: Merrow Water's monthly median days to remediate known-exploited vulnerabilities on internet-facing
# systems, reported to the board, and the age of the evidence behind six assurance claims.
from datetime import date
MONTHS = ["Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec","Jan2","Feb2","Mar2"]
DAYS = [ 12, 9, 14, 11, 13, 10, 12, 15, 11, 9, 13, 24, 21, 23, 25]
BASE = 11 # first 11 months form the baseline
x = DAYS[:BASE]; mean = sum(x) / BASE
mr = [abs(b - a) for a, b in zip(x, x[1:])]; mrbar = sum(mr) / len(mr)
ucl, lcl = mean + 2.66 * mrbar, max(0, mean - 2.66 * mrbar)
print(f"baseline mean {mean:.2f} days, mean moving range {mrbar:.2f}, limits [{lcl:.2f}, {ucl:.2f}]")
run = 0
for m, d in zip(MONTHS, DAYS):
flag = []
if d > ucl or d < lcl: flag.append("outside limits")
run = run + 1 if d > mean else 0
if run >= 4: flag.append(f"{run} in a row above mean")
print(f" {m:5s} {d:3d} {'; '.join(flag)}")
TODAY = date(2026, 10, 12)
CLAIMS = [ # (claim, evidence type, date of evidence, maximum acceptable age in days)
("offline backups restorable", "restore test record", date(2026, 3, 2), 180),
("vendor sessions all brokered", "broker log reconciliation", date(2026, 9, 20), 31),
("no default credentials on PLCs", "engineer attestation", date(2025, 11, 5), 365),
("manual dosing drill passed", "drill report", date(2026, 6, 18), 365),
("IT/OT firewall rules reviewed", "rule review sign-off", date(2026, 1, 9), 180),
("MFA on all remote access", "IdP configuration export", date(2026, 10, 1), 90),
]
print("evidence freshness:")
for c, ev, d, mx in CLAIMS:
age = (TODAY - d).days
print(f" {'STALE' if age > mx else 'ok '} {age:4d}d / {mx:3d}d {c} ({ev})")
Expected output:
baseline mean 11.73 days, mean moving range 3.10, limits [3.48, 19.97]
Jan 12
Feb 9
Mar 14
Apr 11
May 13
Jun 10
Jul 12
Aug 15
Sep 11
Oct 9
Nov 13
Dec 24 outside limits
Jan2 21 outside limits
Feb2 23 outside limits; 4 in a row above mean
Mar2 25 outside limits; 5 in a row above mean
evidence freshness:
STALE 224d / 180d offline backups restorable (restore test record)
ok 22d / 31d vendor sessions all brokered (broker log reconciliation)
ok 341d / 365d no default credentials on PLCs (engineer attestation)
ok 116d / 365d manual dosing drill passed (drill report)
STALE 276d / 180d IT/OT firewall rules reviewed (rule review sign-off)
ok 11d / 90d MFA on all remote access (IdP configuration export)
Interpretation. Over the eleven-month baseline the indicator averaged 11.7 days with natural limits of about 3.5 to 20.0 days. December's 24 days is outside the limits — a signal, not just a bad month — and the following three months are also outside, with a run above the mean. Something changed in December; candidate causes include a staff departure, a change in the vulnerability catalogue's content, a new class of internet-facing systems, or a change in how the indicator is computed. The chart tells the board that to ask, not which answer is right. On evidence, two of six claims are stale: the offline-backup restore test (224 days old against a 180-day limit) and the IT/OT firewall review (276 days against 180). The PLC default-credential claim is within its age limit but rests on an engineer's attestation — the weakest evidence grade — and expires within a month. The board should hear three things: a real deterioration in remediation time since December with an investigation under way; two stale assurance claims with dates for renewal; and one claim that needs stronger evidence, not just newer.
9.9 Common errors and limits
- Reporting activity counts rather than consequence-related indicators.
- Reacting to every monthly rise, or explaining away a real shift as noise.
- Including the suspect period in the baseline, which widens limits and hides the signal.
- Writing assurance claims with no named evidence, or evidence with no date.
- Treating fresh but weak evidence as strong.
- Letting the security function own operational risk on operations' behalf.
- Presenting only good news to the board.
The limit of indicators and charts is that they summarize. They tell a leader where to look; the looking — conversations with operators, site visits, reading incident reviews — remains the leader's job.
Exercises — chapter 9
EX041 — A contaminated baseline. Objective: see how baseline choice affects signals. Inputs: Lab 9. Constraints: change the baseline to the first twelve months (including December). Steps: (1) rerun; (2) report the new mean, moving range and limits; (3) report which later months are now outside the limits; (4) explain the lesson. Expected evidence: output and paragraph. Rubric: 3 = mean 12.75, mean moving range about 3.82, limits about [2.6, 22.9]; December still outside but January (21) now inside; lesson that including the period under test widens limits and can hide a shift; 2 = numbers right, lesson missing; 1 = not run; 0 = none.
EX042 — One-page board report. Objective: report in service units for decisions. Inputs: Lab 9 outputs, Chapter 2 verified coverage for Merrow Water (39%), Chapter 6 evaluation, Chapter 7 RACI findings. Constraints: one page; at most six indicators; end with decisions requested. Steps: write the report. Expected evidence: report. Rubric: 3 = appetite status in service units, signal reported with uncertainty about cause, stale and weak evidence identified, decisions specific with owners; 2 = activity metrics included; 1 = reassurance without evidence; 0 = none.
EX043 — Assurance case. Objective: build a structured argument. Inputs: the claim "Hollin can supply safe water for 72 hours without its control system". Constraints: at least four sub-claims; each with evidence type, maximum age and current status; at least three rebuttals. Steps: build the case as a table or tree. Expected evidence: assurance case. Rubric: 3 = sub-claims cover people, materials, instruments and duration; evidence verified-grade where possible with ages; rebuttals realistic (night shift, analyser power, concurrent incident) with responses; 2 = evidence undated; 1 = restates the claim; 0 = none.
EX044 — Three lines in practice. Objective: assign assurance roles. Inputs: the six assurance claims in Lab 9. Constraints: use the Three Lines Model. Steps: for each claim assign first-line owner, second-line challenge activity and third-line test method; identify one claim where the second line is currently doing first-line work. Expected evidence: table and paragraph. Rubric: 3 = owners are operational roles, challenge and test methods distinct, a credible example of role confusion (for example the security team running restore tests that operations should own); 2 = lines confused in one row; 1 = generic; 0 = none.
EX045 — Capability and succession. Objective: reduce single-person dependence. Inputs: Merrow Water's only engineer who understands the Hollin dosing controller logic plans to retire in 18 months. Constraints: budget for one additional post; operations must not be disrupted. Steps: (1) identify what knowledge is at risk and how to capture it; (2) design a development plan for two successors; (3) define evidence that succession has worked. Expected evidence: plan. Rubric: 3 = knowledge captured as verified artefacts (annotated logic, test procedures, recorded walkthroughs), paired working and supervised changes, evidence such as a successor-led change and drill without the retiring engineer; 2 = training courses only; 1 = documentation only; 0 = none.
Chapter synthesis
Leading security for an essential service means setting direction tied to consequence, building capability that survives departures, and keeping the trust of operations and the board. Report to the board in service units with few indicators and explicit decisions. Use process-behaviour charts to tell change from noise, with a clean baseline. Build assurance cases that connect each claim to dated evidence of known grade, and report stale or weak evidence as gaps. Use the three lines to keep ownership with operations, challenge with specialists and independent testing with audit.