Chapter 14 — Software engineering | Engineering OT, IoT and cyber-physical security

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 14 published. Map, example, EX040.

flowchart TD
  N0["Learning outcomes"]
  N1["14.1 Tools are products"]
  N2["14.2 Gate widening"]
  N3["14.3 CI fixtures"]
  N4["Practice note"]
  N5["14.1b Release trains vs hotfixes"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX040"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["14.1 Tools are products"]
  N2["14.2 Gate widening"]
  N3["14.3 CI fixtures"]
  N4["Practice note"]
  N5["14.1b Release trains vs hotfixes"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX040"]

[Learning outcomes]
    |
    v
[14.1 Tools are products]
    |
    v
[14.2 Gate widening]
    |
    v
[14.3 CI fixtures]
    |
    v
[Practice note]
    |
    v
[14.1b Release trains vs hotfixes]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX040]

Learning outcomes

You will (1) put OT security tools under version control with review; (2) gate conduit-widening changes; (3) keep CI on synthetic fixtures only.

14.1 Tools are products

Allow-list compilers, reachability checkers and detector configs are software. They need repositories, reviews, tags and changelogs. A work permit that cites the firewall export without a hash is incomplete.

Lena keeps CNHP's tools in a single repository with tagged releases and a changelog, and every work permit cites the tag and the hash of the exports it relies on. When a reviewer asks which version of the reachability checker produced a result, the answer is a tag, not a memory.

14.2 Gate widening

Merges that add a conduit, open a function code or remove a deny require a second reviewer and a linked authorization ticket. Tightening changes can be lighter. Encode this in CODEOWNERS on the policy paths.

Lena assigns code owners from control engineering to the allow-list and conduit files, so widening changes cannot merge without them. Tightening changes still need one reviewer. The rule is enforced by code-owner settings on the policy paths, so it cannot be skipped by a busy reviewer. A widening change also needs a linked authorization ticket naming who approved the new path and why.

14.3 CI fixtures

CI runs labs and unit tests on synthetic data. Never point CI at a plant VPN. Seed data lives in the repo; plant credentials live in a vault and never in CI logs.

Synthetic case — Ashford Batch Plant. A PR added a temporary L4 to L2 rule for a vendor visit. CI's reachability check failed because L1-SIS became reachable; the PR was blocked until the rule was rewritten through the DMZ.

Lena's pipeline loads synthetic fixtures for ABP's zones and allow-list.

14.4 Errors and limits

Tags can drift from what operators run; record the running hash in the engagement log at window start.

Lena's engagement log records the hash of the monitor binary and allow-list actually running at the start of each window. If it differs from the tagged release, the window waits until the difference is explained. That check found one bench monitor still running a release candidate after a hotfix. Running hashes go in the window's record. A mismatch is treated as a stop, not a warning.

Practice note

Policy repos use the same dual-approval culture as PLC logic downloads. If downloading ladder to PLC-A needs two people, publishing a conduit that reaches PLC-A should too.

14.1b Release trains vs hotfixes

Normal improvements ride a release train with Lab I/II. A hotfix to the monitor or an allow-list may skip features but not verification: the minimal fail-closed regression pack runs before any window uses it. Tag hotfixes clearly so a window can pin either the train build or the hotfix SHA with eyes open. Version pins appear in the kickoff checklist beside the work-permit hash; if the pin and the running binary disagree, the window does not start.

ABP's hotfixes carry their own tag and the regression pack result in the release note, so a window lead can see exactly what was verified.

14.5 Worked example — reviewing a change to permit() (synthetic ABP)

A pull request widens the HMI's writable range on PLC-A from 40001–40010 to 40001–40012 for a new agitator setpoint. Lena's review checklist: the requirement ID that justifies 40011–40012; new conformance vectors for 40012 (pass) and 40013 (block); unchanged default-deny for unknown sources; a version bump and changelog line; and a second reviewer from control engineering who confirms the registers' meaning in the PLC project. The merge waits for the bench run of the vectors against the actual firewall build. Software engineering discipline here is not ceremony: a one-character range error would let the HMI write a register the process design never intended.

Exercises

EX040 — Pin. Write the method sentence that pins the allow-list for a window. Rubric: 3 = name, hash/tag, date loaded; 2 = name+tag; 1 = name; 0 = none.
EX041 — Gate. One-sentence merge policy for conduit files. Rubric: 3 = second reviewer + AUTH for widen; 2 = reviewer only; 1 = vague; 0 = none.
EX042 — CI. Why must CI never use plant VPN credentials? Rubric: 3 = unauthorized automation, secret leakage, non-determinism; 2 = one solid reason; 1 = vague; 0 = wrong.

Synthesis

Treat OT security software like production software: versions, review gates, synthetic CI.

Continue studying — chapter 14 of 50

← Chapter 13 (draft) · Course chapter index · Chapter 15 (draft) →