OpenCyber Academy
Technologist · Chapter 29 · English
Course code: OCA-092Chapter 29: OCA-092-C029

Chapter 29 — Capstone build

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 29 published. Map, example, EX085.

flowchart TD
  N0["Learning outcomes and connection"]
  N1["29.1 What the capstone must demonstrate"]
  N2["29.2 The build"]
  N3["29.3 Reading the evidence record"]
  N4["29.4 A defect that only appears in combination"]
  N5["29.5 Building the real capstone"]
  N6["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX085"]

Logical drawing

flowchart LR
  N0["Learning outcomes and connection"]
  N1["29.1 What the capstone must demonstrate"]
  N2["29.2 The build"]
  N3["29.3 Reading the evidence record"]
  N4["29.4 A defect that only appears in combination"]
  N5["29.5 Building the real capstone"]
  N6["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX085"]

[Learning outcomes and connection]
    |
    v
[29.1 What the capstone must demonstrate]
    |
    v
[29.2 The build]
    |
    v
[29.3 Reading the evidence record]
    |
    v
[29.4 A defect that only appears in combination]
    |
    v
[29.5 Building the real capstone]
    |
    v
[Chapter synthesis]
    |
    v
[Worked example] --> [Exercise EX085]

Learning outcomes and connection

After this chapter you should be able to:

This is the course's practical outcome: implement an end-to-end service with monitored controls. Lab 29 combines, in one file, the station sealing of Chapter 15, the validation of Chapter 3, a historian in a relational model (Chapter 4), monitoring with an absence rule (Chapter 19) and an evidence record with a digest (Chapters 6 and 20). It is small enough to read in full; the capstone project asks you to build the same shape at larger scale.

29.1 What the capstone must demonstrate

  1. Data path works: readings from all stations reach the historian, validated and authenticated.
  2. Controls work: injected faults — a replay, a forged message, an out-of-range value and a station silence — are each rejected, quarantined or alerted.
  3. Monitoring works: alerts are raised for each injected fault and for nothing else.
  4. Evidence exists: a machine-readable record of the run, with counts that add up and a digest so that later changes to it are detectable.
  5. Limits are stated: what the run does not show.

29.2 The build

Lab 29 generates two hours of synthetic readings from three stations, with PS-09 silent from minute 50 to 84. It injects four tests: a replay of an earlier PS-04 message, a message from PS-07 sealed with the wrong key, and an out-of-range chlorine reading of 7.5 mg/L from PS-07 that takes the slot of minute 20, plus the PS-09 silence. The relay verifies the MAC, rejects replays, quarantines out-of-range values and stores the rest; monitoring adds a silence rule; the evidence record summarizes the run.

Lab 29 (save as lab29.py, run with python3 lab29.py):

# Lab 29 - capstone: an end-to-end monitored telemetry service in one file (offline, standard library only)
# Synthetic stations -> authenticated relay -> validated historian (sqlite3) -> monitoring -> evidence record.
import hashlib, hmac, json, random, sqlite3
random.seed(29)
KEY = b"lantern-lab-key-not-real-0001"
LIMITS = {"LEVEL_M": (0.0, 12.0), "CL2_MGL": (0.0, 5.0)}
def seal(m, key=KEY):
    body = json.dumps(m, sort_keys=True, separators=(",", ":"))
    return body + "." + hmac.new(key, body.encode(), hashlib.sha256).hexdigest()
# 1. stations emit one reading per minute for 120 minutes; PS-09 goes silent 50-84; one injected test of each fault
wire, now0 = [], 1_790_000_000
for st, tag, base in (("PS-04", "LEVEL_M", 3.4), ("PS-07", "CL2_MGL", 0.6), ("PS-09", "LEVEL_M", 5.1)):
    for i in range(120):
        if st == "PS-09" and 50 <= i < 85: continue
        if st == "PS-07" and i == 20: continue  # the injected out-of-range test takes this slot
        wire.append((now0 + 60*i, seal({"st": st, "tag": tag, "v": round(base + random.gauss(0, 0.05), 3), "ts": now0 + 60*i, "seq": i})))
wire.append((now0 + 600, wire[3][1]))                                                         # TEST: replay
wire.append((now0 + 900, seal({"st": "PS-07", "tag": "CL2_MGL", "v": 0.6, "ts": now0 + 900, "seq": 999}, key=b"x")))  # TEST: forged
wire.append((now0 + 1200, seal({"st": "PS-07", "tag": "CL2_MGL", "v": 7.5, "ts": now0 + 1200, "seq": 20})))         # TEST: out of range
wire.sort(key=lambda w: w[0])
# 2. relay + historian
db = sqlite3.connect(":memory:")
db.execute("CREATE TABLE reading(st TEXT, tag TEXT, v REAL, ts INTEGER, seq INTEGER, PRIMARY KEY(st, seq))")
events, last = [], {}
for t, w in wire:
    body, mac = w.rsplit(".", 1)
    if not hmac.compare_digest(hmac.new(KEY, body.encode(), hashlib.sha256).hexdigest(), mac):
        events.append((t, "mac_reject", "?")); continue
    m = json.loads(body)
    if m["seq"] <= last.get(m["st"], -1):
        events.append((t, "replay_reject", m["st"])); continue
    lo, hi = LIMITS[m["tag"]]
    last[m["st"]] = m["seq"]
    if not lo <= m["v"] <= hi:
        events.append((t, "range_quarantine", m["st"])); continue
    db.execute("INSERT INTO reading VALUES (?,?,?,?,?)", (m["st"], m["tag"], m["v"], m["ts"], m["seq"]))
# 3. monitoring: security events and silence > 15 min
alerts = [(t, kind, st) for t, kind, st in events]
for st, in db.execute("SELECT DISTINCT st FROM reading"):
    ts = [r[0] for r in db.execute("SELECT ts FROM reading WHERE st=? ORDER BY ts", (st,))]
    alerts += [(a + 900, "silence", st) for a, b in zip(ts, ts[1:]) if b - a > 900]
# 4. evidence record
stored = db.execute("SELECT COUNT(*) FROM reading").fetchone()[0]
record = {"period_utc_s": [now0, now0 + 7140], "messages_on_wire": len(wire), "stored": stored,
          "alerts": sorted({(k, s) for _, k, s in alerts}), "injected_tests": 4,
          "injected_tests_detected": len({k for _, k, _ in alerts} & {"replay_reject", "mac_reject", "range_quarantine", "silence"}),
          "alert_count": len(alerts)}
blob = json.dumps(record, sort_keys=True).encode()
print(json.dumps(record, sort_keys=True))
print("evidence sha256:", hashlib.sha256(blob).hexdigest()[:24], "...")
print("PS-07 mean chlorine stored: %.3f mg/L" % db.execute("SELECT AVG(v) FROM reading WHERE st='PS-07'").fetchone()[0])

Expected output (Python 3.13.5):

{"alert_count": 4, "alerts": [["mac_reject", "?"], ["range_quarantine", "PS-07"], ["replay_reject", "PS-04"], ["silence", "PS-09"]], "injected_tests": 4, "injected_tests_detected": 4, "messages_on_wire": 327, "period_utc_s": [1790000000, 1790007140], "stored": 324}
evidence sha256: aa40e3905d178699f1460e1c ...
PS-07 mean chlorine stored: 0.606 mg/L

29.3 Reading the evidence record

Check that the numbers add up. Messages on the wire: 120 + 119 + 85 + 3 injected = 327. Stored: 324 (all genuine readings). Not stored: 3 (the replay, the forgery and the out-of-range reading). Alerts: four, one of each kind, and four of four injected tests detected. The digest lets anyone confirm later that the record has not been edited; in a real service the record would be stored off-host and the digest written into the change or test ticket.

If the stored count had read 325, one genuine reading would be unexplained.

29.4 A defect that only appears in combination

Change the injected out-of-range message's sequence number from 20 to 1000 — as a faulty station gateway might after a reset — and rerun. The result changes dramatically: stored readings fall from 324 to 225 and alerts rise from 4 to 103. Ninety-nine genuine PS-07 readings are rejected as replays. The cause is that the relay updates its last-seen sequence number after authentication but before the range check: the out-of-range message is authentic, so its sequence number 1000 is recorded, and every later genuine message (sequence 21 to 119) looks like a replay.

Each component was "correct" by its own chapter's tests. Chapter 3 warned against updating replay state before all other checks have passed; Chapter 15 put the update after authentication. The combination shows the remaining hole: an authentic but implausible message can poison the replay state. Fixes include: update the state only on accept or quarantine of a message that is plausible in sequence (reject sequence jumps larger than a bound, say 1,000 or one day of messages, unless in an explicit resynchronization mode); alert on large jumps; and give operators a reviewed resynchronization procedure. This is exactly why the capstone tests the integrated service, not just its parts.

29.5 Building the real capstone

The project version of the capstone, done in a team (Chapter 25), builds the same shape with real components in a lab: three relay virtual machines, a historian database, log forwarding to a collector, the normalizer, three monitoring rules plus self-monitoring, configuration as code with a drift check, a backup and timed restore, runbooks and a handover pack. Use only lab equipment or simulated stations. The deliverable is the service evidence record: for each requirement in "What the capstone must demonstrate" above, the test performed, its date, the raw output, the result against a pre-stated claim rule (Chapter 23), and the open items.

29.6 Common errors and limits

Team A's summary in Chapter 30 commits the fifth: it states no limits at all.

An injected forgery built with the station's real key would test nothing; the team must design tests that could fail.

Exercises — chapter 29

EX085 — Reproduce the defect. Objective: find and explain an integration defect. Inputs: Lab 29. Constraints: change only the sequence number of the out-of-range test. Steps: (1) set it to 1000 and run; (2) report stored count, alert count and which messages were lost; (3) explain the cause in terms of the order of checks. Expected evidence: output and explanation. Rubric: 3 = stored 225, alert_count 103, 99 genuine PS-07 readings rejected as replays, explanation that state was updated for an authentic but out-of-range message; 2 = numbers right, explanation thin; 1 = not run; 0 = none.

EX086 — Fix it. Objective: correct the defect without opening a replay hole. Inputs: Lab 29 with sequence 1000. Constraints: standard library only; replays must still be rejected. Steps: (1) implement a maximum sequence jump of 1,000 (reject larger jumps with a distinct event seq_jump_reject) or another defensible fix; (2) run with sequence 1000 and with sequence 20; (3) show that the replay test is still rejected in both. Expected evidence: code and both outputs. Rubric: 3 = jump of 981 (19→1000) is within 1,000 — so the learner must notice that a bound of 1,000 does not catch this case and choose a tighter or time-based bound (for example jump no more than the elapsed minutes plus a margin), with stored 324 restored and replay still rejected; 2 = fix works for this case but reasoning about the bound missing; 1 = removes replay protection; 0 = none.

EX087 — Evidence record review. Objective: check a record as an assessor. Inputs: the Lab 29 output with sequence 20. Constraints: checks only from the record and the lab source. Steps: (1) verify the counts add up; (2) recompute the digest from the printed JSON and compare its first 24 characters; (3) list three limits the record should state. Expected evidence: calculations and list. Rubric: 3 = 327 = 324 + 3 confirmed; digest recomputed by hashing the exact JSON string; limits such as synthetic data, no field devices or carrier, single run, no adaptive adversary, no load at scale; 2 = two parts; 1 = one; 0 = none.

Chapter synthesis

The capstone integrates the course into one monitored service and proves it with injected tests and a checkable evidence record. Counts must add up, the record must be protected by a digest and stored off-host, and its limits must be stated. Integration testing finds defects that component tests miss: here an authentic but implausible message poisoned replay state, a hole visible only when validation, authentication and ordering met.

Worked answers

EX085 — Reproduce the defect. With the out-of-range test's sequence set to 1000: "stored": 225, "alert_count": 103, alerts now include ["replay_reject", "PS-07"]. Lost: 99 genuine PS-07 readings (sequence 21–119). Cause: the relay records the last sequence number after authentication but before the range check; the authentic out-of-range message sets PS-07's last sequence to 1000, so every later genuine message has a lower number and is rejected as a replay.

EX086 — Fix it. A bound of 1,000 does not help here: the jump from 19 to 1000 is 981. A time-based bound works: reject (event seq_jump_reject) when the jump exceeds the minutes elapsed since the station's last accepted message plus a margin of 5. Implemented in Lab 29 (track last_t per station alongside last), the run with sequence 1000 gives "stored": 324, "alert_count": 4 with seq_jump_reject for PS-07 in place of the range quarantine, and the PS-04 replay is still rejected; with sequence 20 the output is unchanged from the original (stored 324, four alerts). PS-09's return after its 35-minute silence is accepted because its jump of 36 is within 36 + 5. Note that injected_tests_detected reports 3 with sequence 1000 because the detection set does not yet include seq_jump_reject — add it.

EX087 — Evidence record review. Counts: messages on wire 327 = 120 (PS-04) + 119 (PS-07, minus the slot taken by the test) + 85 (PS-09) + 3 injected messages; stored 324 = 327 − 3 (replay rejected, forged rejected, out-of-range quarantined). Digest: hashing the printed JSON line exactly (sha256 of the bytes without the trailing newline) gives a digest starting aa40e3905d178699f1460e1c, matching the printout. Limits to state: synthetic stations and values; one two-hour run; no real field devices, carrier or load at scale; no adaptive adversary; key handling simplified (key in source).