Chapter 11 — Critical infrastructure and sector security: verification
Lesson map
Lesson 11 published. Map, example, EX041.
flowchart TD
N0["Learning outcomes and connection"]
N1["11.1 Requirements that can be tested"]
N2["11.2 Positive and negative tests"]
N3["11.3 Evidence rules"]
N4["11.5 Safe verification in control networks"]
N5["11.6 Practical activity"]
N6["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX041"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["11.1 Requirements that can be tested"]
N2["11.2 Positive and negative tests"]
N3["11.3 Evidence rules"]
N4["11.5 Safe verification in control networks"]
N5["11.6 Practical activity"]
N6["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX041"]
[Learning outcomes and connection]
|
v
[11.1 Requirements that can be tested]
|
v
[11.2 Positive and negative tests]
|
v
[11.3 Evidence rules]
|
v
[11.5 Safe verification in control networks]
|
v
[11.6 Practical activity]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX041]
Learning outcomes and connection
After this chapter you should be able to:
- write requirements as testable statements and link each to at least one test (traceability);
- design positive tests (the needed function still works) and negative tests (the forbidden action fails) for every control change;
- refuse to count a test without evidence, and report requirements as verified, failed or untested;
- verify without harming the process: small, scoped tests in agreed windows.
Chapter 10 changed configuration. A change is not finished when it is applied; it is finished when its requirements are shown to hold. This chapter makes that showing systematic.
11.1 Requirements that can be tested
"The historian is hardened" cannot be tested. "SSH on hist-02 listens only on 10.90.0.21" can. Write each requirement so that a named test, run by someone else, could fail it. Split compound requirements: "only the SCADA server and engineering workstation reach station tcp/502" needs a positive test (SCADA still polls) and a negative test (another control-centre host is refused).
REQ-1 in Lab 11 passes this check: SSH listening only on 10.90.0.21 is something tests T1 and T2 could have failed. REQ-4 shows the split in practice: T5 and T6 cover its negative and positive halves, and the requirement fails if either half fails. Put the requirement identifiers in the change record before the work starts.
11.2 Positive and negative tests
Negative tests show a control blocks what it should; positive tests show it does not block what operations need. A firewall change that passes all its negative tests but stops historian collection has failed. In control environments the positive tests matter as much as the negative ones: availability is usually the first priority. NIST's testing guide discusses planning and executing technical tests (SP 800-115); continuous monitoring guidance stresses that control effectiveness must be re-shown over time, not assumed (SP 800-137).
11.3 Evidence rules
A test counts only if it has: an identifier, the requirement it verifies, the method, the time, the person, the result and an evidence file (Chapter 5). A "config review only" with no recorded output is a useful activity but not a test result. A requirement with no counted test is untested — not verified, and not failed. Reporting untested as verified is the single most common way verification misleads.
Lab 11 applies the rule to T8, a config review of REQ-2 with no evidence file, which the script ignores. REQ-6 in Lab 11 stays untested until someone produces a deny line from the collector; reporting it as verified because the rule exists would hide the gap. Keep evidence files next to the test identifiers in the verification record.
11.4 Worked example: verifying CHG-2207 (synthetic)
CHG-2207 hardened SSH on hist-02 and was meant to close corporate access to the control centre on fw-01. The verification record:
Lab 11 (save as lab11.py, run with python3 lab11.py):
# Lab 11 - requirement-to-test traceability after a hardening change (offline, stdlib)
# Synthetic verification record for change CHG-2207 on HIST-02 and fw-01, 2026-10-12.
requirements = {
"REQ-1": "SSH listens only on management address 10.90.0.21",
"REQ-2": "Root cannot log in over SSH",
"REQ-3": "Corporate desks cannot reach control-centre RDP",
"REQ-4": "Only SCADA server and engineering workstation reach station tcp/502",
"REQ-5": "Historian still receives KW-01..KW-12 readings after change",
"REQ-6": "Firewall denies are logged to the collector",
}
tests = [ # test id, requirement, method, positive/negative, result, evidence
("T1", "REQ-1", "listening-socket listing on host", "neg", "pass", "ss-output-0612.txt"),
("T2", "REQ-1", "connect to 10.20.0.21:22 from jump host", "neg", "pass", "conn-0614.txt"),
("T3", "REQ-2", "root login attempt from jump host", "neg", "pass", "auth-0615.log"),
("T4", "REQ-3", "RDP connect from 10.30.3.7", "neg", "fail", "conn-0618.txt"),
("T5", "REQ-4", "Modbus read from 10.20.0.12", "neg", "fail", "conn-0620.txt"),
("T6", "REQ-4", "Modbus read from SCADA 10.20.0.10", "pos", "pass", "poll-0621.txt"),
("T7", "REQ-5", "count stations reporting 06:30-06:40Z", "pos", "pass", "hist-count.csv"),
("T8", "REQ-2", "config review only", "neg", "pass", ""),
]
status = {}
for tid, req, method, kind, res, ev in tests:
if not ev: print(f"{tid} {req} ignored: no evidence file recorded ({method})"); continue
status.setdefault(req, []).append((tid, kind, res))
for req, text in requirements.items():
rs = status.get(req, [])
if not rs: verdict = "UNTESTED"
elif any(r == "fail" for _, _, r in rs): verdict = "FAILED"
else: verdict = "verified"
kinds = "+".join(sorted({k for _, k, _ in rs})) or "-"
print(f"{req} {verdict:<9} tests={','.join(t for t, _, _ in rs) or '-':<6} kinds={kinds:<7} {text}")
v = sum(1 for r in requirements if status.get(r) and all(x[2] == "pass" for x in status[r]))
print(f"verified {v}/{len(requirements)}; the change is not complete until every requirement is verified")
Expected output (Python 3.13.5):
T8 REQ-2 ignored: no evidence file recorded (config review only)
REQ-1 verified tests=T1,T2 kinds=neg SSH listens only on management address 10.90.0.21
REQ-2 verified tests=T3 kinds=neg Root cannot log in over SSH
REQ-3 FAILED tests=T4 kinds=neg Corporate desks cannot reach control-centre RDP
REQ-4 FAILED tests=T5,T6 kinds=neg+pos Only SCADA server and engineering workstation reach station tcp/502
REQ-5 verified tests=T7 kinds=pos Historian still receives KW-01..KW-12 readings after change
REQ-6 UNTESTED tests=- kinds=- Firewall denies are logged to the collector
verified 3/6; the change is not complete until every requirement is verified
Three requirements verified, two failed, one untested. REQ-3 failed because R4 was left in place: the ticket said "fix the RDP rule", and the engineer corrected R5 rather than the rule that shadowed it — a common pattern when a change is written as a line edit instead of as a requirement. REQ-4 failed its negative test (10.20.0.12 can still read stations, via R6) while its positive test passed — exactly why both kinds are needed. REQ-6 has no test at all; nobody checked that denies reach the collector, so the detections of Chapter 17 would be blind to denied attempts. T8 was ignored because it has no evidence file. The change goes back to "in progress" with three open items, each with an owner.
11.5 Safe verification in control networks
- Prefer reading results the system already records (firewall logs, historian counts) over generating traffic.
- When generating traffic, use a marked test source (for example 10.30.99.9), the smallest possible action (a TCP connect with no payload), and an agreed window with operations informed.
- Never verify a write-block by attempting a write to a live controller; verify it with a connect test and the firewall log, or against a spare device in a lab (Chapter 12).
- Stop at the first unexpected behaviour.
Operations should know before the first connect that 10.30.99.9 is the test source, so its traffic is not mistaken for an intruder.
11.6 Practical activity
Write a test for REQ-6 that does not touch a station: a connect attempt from the marked test desk to a control-centre port that should be denied, then a query on the collector for the matching deny line within 60 s. Add it to Lab 11 as T9 with an evidence file name and re-run.
Use the marked test desk, then query the collector for the matching deny within sixty seconds. Add the test as T9 with an evidence file name and re-run Lab 11 so that REQ-6 moves from untested to verified or failed. Keep the collector query text in the evidence file so the test can be repeated.
11.7 Common errors and limits
- Running only negative tests and missing that historian collection stopped.
- Counting a test without evidence.
- Reporting "5 of 6 tests passed" instead of the status of each requirement.
- Verifying once and assuming forever; re-verify after related changes.
-
Lab 11 treats any failing test as a failed requirement; a requirement can also be partly verified if a test covers only some of its scope — say so in words.
-
Leaving REQ-6 untested because producing a deny line at the collector seemed harder than reading the firewall rule.
Exercises — chapter 11
EX041 — Make it testable. Objective: rewrite requirements. Inputs: "Remote access is secure", "Time is synchronized", "Logs are kept". Steps: rewrite each as one or more testable requirements for KVROC. Expected evidence: rewritten requirements with a test idea each. Rubric: 3 = specific statements such as "vendor access to SS-3 is possible only via jump-01 with MFA during an enabled window", "all Z2 hosts within 1 s of ntp-ops", "fw-01 allow and deny events reach the collector within 60 s and are retained 12 months"; each with a test; 2 = one vague; 1 = restated; 0 = none.
EX042 — Add the missing test. Objective: close an untested requirement. Inputs: Lab 11 and the activity in 11.6. Steps: add T9 for REQ-6 with result pass and evidence file collector-0630.txt; run. Expected evidence: output. Rubric: 3 = REQ-6 verified via T9, total "verified 4/6"; explains that REQ-3 and REQ-4 still block completion; 2 = output only; 1 = attempted; 0 = none.
EX043 — Positive tests for a firewall change. Objective: protect availability. Inputs: the fw-01 fix from EX038. Steps: list the positive tests needed before closing the change. Expected evidence: list with method and evidence. Rubric: 3 = SCADA polls all twelve stations (historian count), engineering workstation can connect to a station in an approved window, jump-01 SSH to Z2 works, hist-02 daily export still arrives; each with evidence source; 2 = some; 1 = none positive; 0 = nothing.
EX044 — Untested is not verified. Objective: report honestly. Inputs: a manager's summary "CHG-2207: 7 of 8 tests passed, done". Steps: write the corrected one-paragraph status. Expected evidence: paragraph. Rubric: 3 = states per-requirement status (3 verified, 2 failed, 1 untested), names the open items and owners, says the change is not complete; 2 = correct but no owners; 1 = minor edit; 0 = none.
Chapter synthesis
Verification turns a configuration change into a demonstrated result. Write requirements that can fail, test each with positive and negative tests, count only tests with evidence, and report each requirement as verified, failed or untested. In control networks, verify with the smallest scoped actions, and never by experimenting on the live process.
Continue studying — chapter 11 of 18
← Chapter 10 · Course chapter index · Chapter 12 →
Continue to the next authored lesson.