Chapter 38 — OT, IoT and cyber-physical security: validation
Lesson map
Lesson 38 published. Map, example, EX112.
flowchart TD
N0["Learning outcomes"]
N1["38.1 Re-test design"]
N2["38.2 Negative results"]
N3["38.3 No new methods"]
N4["Practice note"]
N5["38.1b Evidence of validation and adjudication"]
N6["Field note — sample re-test"]
N7["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX112"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["38.1 Re-test design"]
N2["38.2 Negative results"]
N3["38.3 No new methods"]
N4["Practice note"]
N5["38.1b Evidence of validation and adjudication"]
N6["Field note — sample re-test"]
N7["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX112"]
[Learning outcomes]
|
v
[38.1 Re-test design]
|
v
[38.2 Negative results]
|
v
[38.3 No new methods]
|
v
[Practice note]
|
v
[38.1b Evidence of validation and adjudication]
|
v
[Field note — sample re-test]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX112]
Learning outcomes
You will (1) validate fixes with re-tests; (2) avoid validation that reintroduces process risk; (3) record negative results.
38.1 Re-test design
Same vantage, least privilege, same evidence standard. For an allow-list fix: confirm the write is denied on the bench; do not spray writes at the live PLC.
When the HIST-1 allow-list was tightened at ABP, Lena re-tested from the same jump host and account used for the original finding. She replayed a function 16 write to the bench copy of PLC-A and confirmed the monitor denied it with the expected reason code. The live check was read-only: she compared the deployed allow-list version with the bench version rather than sending writes toward production equipment.
38.2 Negative results
Still open is a valid validation outcome. Finding stores must accept it without forcing close.
At ABP the re-test of shared engineering passwords found they were still in use on two workstations, so the finding stayed open with new evidence attached. Lena's finding store allows a re-test to record still open with a date and a reason, and the owner receives a new deadline. Forcing every re-test to close a finding would encourage reviewers to mark fixes done when they are only partly applied.
38.3 No new methods
Validation inside a CNHP window uses the method that found the issue, or a weaker read-only check such as a passive monitor capture — not a chance to try new attacks.
Synthetic case — Ashford Batch Plant. Re-test of the L4 to L2 rule: reachability now excludes L1-SIS; ticket closed with export hash. A separate finding on shared eng passwords stayed open.
Lena's validation plan names the original method in the ticket before the re-test starts. When a reviewer suggested trying a new fuzzing tool during validation, she logged it as a proposal for the next assessment instead.
38.4 Errors and limits
Different environment (bench vs plant) means you validated the bench.
At ABP, Lena's allow-list fix was first validated on the bench copy of PLC-A, and the ticket said so. A separate read-only check confirmed that the production monitor was running the same allow-list version, and only then was the finding marked fixed in production. Reports keep the two results apart, because a bench pass cannot show that the plant's configuration matches the bench, and some differences, such as firmware revisions, would change the result.
Practice note
Automated re-tests are allow-listed checks with hard timeouts. They never include credential stuffing or exploit retries.
38.1b Evidence of validation and adjudication
CNHP validation tickets link to the original finding, the method used, the work-permit number, the timestamp and the hash of the new evidence. A supervisor's "looks fine" without those fields does not close a critical finding. Disputes escalate to a named adjudicator who compares evidence hashes — they do not re-attack.
When many similar findings share a root cause, re-test a sample plus the originally reported assets, and say so. Adjudicator decisions are final for the window and may only be appealed in the retrospective.
Lena's tickets carry all four fields.
Field note — sample re-test
Twenty HMIs shared a missing allow-list entry. Validation re-tested five plus the originally reported two, documented the sample, and left the finding open until the fleet job finished. Claiming all twenty closed after two checks would have been a measurement lie.
38.5 Worked example — validation re-test of the zoning design (synthetic ABP)
Six months after deployment, Lena re-runs the reachability check from Chapter 2 against the live firewall export and the passive-discovery conduit list. The export matches the design for 8 of 9 conduits; the ninth shows an undocumented L4→L3 rule added during an ERP upgrade. The validation report records the negative finding with evidence, assigns it to the network owner, and does not introduce any new test method mid-validation, so results remain comparable with the commissioning baseline. The SIS-B zone remains unreachable in both datasets, which is reported as confirmed rather than assumed.
Exercises
- EX112 — Re-test. Steps to validate a removed L4 to L2 rule safely. Rubric: 3 = export+reachability, expect no path, evidence, no live writes; 2 = partial; 1 = live attack; 0 = none.
- EX113 — Negative. Why must still open be storable? Rubric: 3 = honesty/metrics; 2 = vague; 1 = always close; 0 = none.
- EX114 — Env. Limitation when re-test was bench-only. Rubric: 3 = clear; 2 = vague; 1 = claims plant; 0 = none.
Synthesis
Validate narrowly; keep negatives; don't widen methods.
Continue studying — chapter 38 of 50
← Chapter 37 (draft) · Course chapter index · Chapter 39 (draft) →