Chapter 3 — OT, IoT and cyber-physical security: specialist evidence
Lesson map
Lesson 03 published. Map, example, EX011.
flowchart TD
N0["Learning outcomes"]
N1["3.1 Sources and their blind spots"]
N2["3.2 Conflicts are information"]
N3["3.3 Appraising external claims"]
N4["3.4 Confidence statements"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX011"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["3.1 Sources and their blind spots"]
N2["3.2 Conflicts are information"]
N3["3.3 Appraising external claims"]
N4["3.4 Confidence statements"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX011"]
[Learning outcomes]
|
v
[3.1 Sources and their blind spots]
|
v
[3.2 Conflicts are information]
|
v
[3.3 Appraising external claims]
|
v
[3.4 Confidence statements]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX011]
Learning outcomes
You will (1) appraise the quality of OT security evidence sources; (2) reconcile conflicting evidence from network, host and process data; (3) assess vendor and advisory claims critically; and (4) write an evidence appraisal that states confidence levels.
3.1 Sources and their blind spots
Passive network captures show who talked to whom but miss serial links and encrypted payload meaning. Controller event logs, where they exist, show program changes but often roll over after a few hundred entries. Process data in the historian shows physical effect but not cause. HMI audit trails show operator actions but not engineering-tool sessions. Specialist evidence work combines them because each misses something.
Before relying on any source, write down what it can and cannot see at BWRP. The capture covers the plant network but not a technician at the dosing panel. PLC-D's event log may have rolled over since the event. The historian records the setpoint but not who changed it. That list tells Daniel which second source to consult for each question.
3.2 Conflicts are information
If the historian shows a dosing setpoint change at 14:02 but neither the HMI audit trail nor the network capture shows a write, the possibilities are a missing capture, a clock offset, a local change at the panel, or logic that changes the setpoint internally. Each hypothesis predicts something checkable: panel key-switch logs, a PLC program comparison, the capture's packet-loss counters.
Write each hypothesis with the check that would rule it out, and run the cheapest checks first. A clock offset can be tested in minutes by comparing timestamps of a known event across systems. Record the result of every check, including the ones that only eliminate a possibility, because they are what make the final conclusion credible.
3.3 Appraising external claims
CISA ICS advisories describe affected products, versions, CVSS scores and mitigations coordinated with vendors; they are strong evidence about a vulnerability's existence, weaker about whether it is exploitable in your configuration. Vendor statements about "not affected" deserve a request for the reasoning. Peer-reviewed measurement studies, such as the Mirai analysis by Antonakakis et al. (USENIX Security 2017), carry method sections you can evaluate.
For BWRP, an advisory affecting a product used on PLC-D should lead to two questions: is the affected version installed, and is the vulnerable function reachable from where an attacker could be. Record the answers next to the advisory in the inventory. When a vendor says not affected, file its reasoning with the date, so the claim can be checked later.
3.4 Confidence statements
State confidence explicitly: high (multiple independent sources agree), moderate (one strong source, no contradiction), low (inference from indirect data). Leaders act differently on each.
Attach the level to each conclusion, not to the report as a whole, and say which sources support it. Daniel's 14:02 finding is high confidence because the capture, the program comparison and the key-switch log agree. A conclusion resting on the key-switch log alone would be moderate, and a guess about who was on site, before checking rosters, would be low.
Worked example (synthetic BWRP)
At 14:02 the historian shows the hypochlorite dosing setpoint on PLC-D moving from 4.0 to 6.5 mg/L. No HMI audit entry, no Modbus write in the capture. Daniel checks: capture loss counters show 0 dropped packets; PLC program comparison shows an unchanged program; the local panel key-switch log shows "local mode 14:01–14:05". Appraisal: high confidence the change was made locally at the panel; next step, identify who was on site.
Synthetic case — Bayside Water Reclamation Plant. Times, tags and values are invented.
Errors and limits
Absence of evidence in a capture is meaningful only if the capture's coverage and loss are known. Logs can be altered by a capable intruder.
Exercises
- EX011 — Predictions from hypotheses. For the hypothesis "clock offset", state what evidence would confirm it. Rubric: 3 = matching write found at offset time in capture or HMI log with measured offset; 2 = partial; 1 = vague; 0 = none.
- EX012 — Confidence for one-source evidence. The only evidence of a remote login is one VPN log line. What confidence and why? Rubric: 3 = moderate at best, single source, could be altered or misattributed; 2 = level only; 1 = high; 0 = none.
- EX013 — Interpreting a "not affected" claim. A vendor says its RTU is not affected by an advisory's library flaw. What do you ask? Rubric: 3 = which version/component, why unreachable, evidence (SBOM, analysis); 2 = one; 1 = accept; 0 = none.
- EX014 — What capture loss counters prove. Why did Daniel check packet-loss counters before trusting "no write seen"? Rubric: 3 = absence only meaningful if capture complete; 2 = partial; 1 = vague; 0 = none.
- EX015 — Blind spot of process data. Name one thing the historian can never tell you about the 14:02 change. Rubric: 3 = who or what initiated it (cause/identity); 2 = vague; 1 = wrong; 0 = none.
Synthesis
Specialist evidence work triangulates sources with known blind spots, treats conflicts as hypotheses to test, and attaches confidence to every conclusion.
Continue studying — chapter 03 of 12
← Chapter 02 (draft) · Course chapter index · Chapter 04 (draft) →