◇ OpenCyber Academy
Doctoral research · Chapter 7 · English

Chapter 7 — Thesis synthesis and reproducibility archive

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 7 — Thesis synthesis and reproducibility archive

Lesson map

Lesson 07 published. Map, example, EX039.

flowchart TD
  N0["Learning outcomes and connection"]
  N1["7.1 Argument map"]
  N2["7.2 Format"]
  N3["7.3 Tiered archive"]
  N4["7.4 Documenting synthetic data and models"]
  N5["7.5 Directional limitations"]
  N6["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX039"]

Logical drawing

flowchart LR
  N0["Learning outcomes and connection"]
  N1["7.1 Argument map"]
  N2["7.2 Format"]
  N3["7.3 Tiered archive"]
  N4["7.4 Documenting synthetic data and models"]
  N5["7.5 Directional limitations"]
  N6["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX039"]

[Learning outcomes and connection]
    |
    v
[7.1 Argument map]
    |
    v
[7.2 Format]
    |
    v
[7.3 Tiered archive]
    |
    v
[7.4 Documenting synthetic data and models]
    |
    v
[7.5 Directional limitations]
    |
    v
[Chapter synthesis]
    |
    v
[Worked example] --> [Exercise EX039]

Learning outcomes and connection

After this chapter you can (1) map the thesis argument from evidence to claim, (2) choose a thesis format, (3) build a tiered, hash-checked archive that fails closed, (4) document synthetic data and models, and (5) write limitations that say which direction each bias pushes. It assembles Chapters 1–6 and prepares the defense.

7.1 Argument map

Use a Toulmin structure: claim (narrowed claim of §6.3), data (Lab 3, Lab 5, T2 results), warrant (the mass balance links flow and level; an attacker controlling only one cannot keep both consistent), backing (verification tests, Page's CUSUM properties), qualifier ("in two synthetic contexts"), rebuttal (class B, slow ramps, laggy instruments). Each element points to a chapter and an artifact.

Draw the map as one page and check each arrow: does the cited data actually support the warrant, and does the rebuttal list every boundary from the critique chapter? An element with no artefact behind it is a gap to close before the defense. Keep the map current as wording changes, because examiners often ask how a single sentence connects to the evidence.

7.2 Format

A monograph suits a single integrated argument; a publication-based thesis suits independent papers with a linking chapter. Either way, the reproducibility archive belongs to the thesis, not to one paper.

For the Setpoint Study a publication-based thesis maps naturally onto the three planned papers, with the linking chapter carrying the narrowed claim and the boundary conditions. A monograph would place the same material in one continuous argument. Choose with your committee early, because the choice changes how much repetition is acceptable and how co-authored work is credited.

7.3 Tiered archive

OT artifacts carry different risks. Code and the synthetic simulator are T0 (public). Synthetic traces are T1. Testbed configurations are T2 (vetted researchers). Plant-derived maps are T3 and never released. A release must fail closed: if any file's hash differs from the manifest, nothing is released; files above the release tier are excluded automatically. Lab 7 demonstrates this logic.

# Lab 7: fail-closed reproducibility manifest check (stdlib). Creates synthetic files in ./archive_demo.
import hashlib, json, os
os.makedirs("archive_demo", exist_ok=True)
files = {"lab3.py": "print('model')\n", "traces_T1.csv": "t,level\n0,50.0\n", "plant_map_T3.txt": "RESTRICTED\n"}
for n, c in files.items():
    open(os.path.join("archive_demo", n), "w").write(c)
manifest = [{"path": n, "tier": "T3" if n.startswith("plant_map") else "T1",
             "sha256": hashlib.sha256(c.encode()).hexdigest()} for n, c in files.items()]
manifest[1]["sha256"] = "0"*64          # simulate a stale hash after an edit
def check(release_tier_max="T1"):
    ok = True
    for e in manifest:
        h = hashlib.sha256(open(os.path.join("archive_demo", e["path"]), "rb").read()).hexdigest()
        if e["tier"] > release_tier_max:
            print("EXCLUDE (tier)", e["path"], e["tier"]); continue
        if h != e["sha256"]:
            print("FAIL (hash)   ", e["path"]); ok = False
        else:
            print("OK            ", e["path"])
    return ok
print("release allowed:", check())

Expected output:

OK             lab3.py
FAIL (hash)    traces_T1.csv
EXCLUDE (tier) plant_map_T3.txt T3
release allowed: False

The stale hash blocks the release, and the T3 file is never considered. The comparison e["tier"] > release_tier_max works because tier names sort lexically T0 < T1 < T2 < T3 — a convenience that would break with a tier named "T10", which is noted in the archive's README.

7.4 Documenting synthetic data and models

State for each synthetic dataset whether it is invented (parameters chosen for teaching) or calibrated (fitted to a real system's statistics). The Setpoint Study's tank is invented. Record parameters, seeds, generator code version and the intended use; warn against using it to make claims about real plants.

Write this documentation as a short data sheet stored with the generator code, and include the equilibrium, the orifice coefficient of 0.02, the noise levels and the attack gains. Name the labs that used each configuration. Anyone reusing the traces should see at once that they describe a teaching model, not measurements from Kestrel or any other plant.

7.5 Directional limitations

A useful limitations table states the direction of each bias: noise-free flow meter in simulation (overstates recall — confirmed by T2), single-loop model (unknown direction for multi-loop plants), synthetic operators (unknown), testbed scale (likely overstates signal-to-noise), censoring at 400 steps (understates Plant B delay cost).

The direction matters because it tells a reader how to adjust confidence. If a bias overstates recall, the real-world figure is likely lower; if it is unknown, the reader should not assume either way. Keep the table beside the results rather than at the end, and update it when new evidence, such as T2's lower recall, settles a direction.

7.6 Common errors and limits

Archives that release everything or nothing; manifests not checked; calling invented data "realistic"; limitations without direction.

An archive that releases everything leaks plant-level detail; one that releases nothing cannot be checked. A manifest that is never verified gives false comfort, which is why Lab 7 fails closed. Synthetic data should be called invented or calibrated, never realistic. And every limitation should say whether it pushes results up, down or in an unknown direction. Run the manifest check on the release copy, never only on the working directory.

Exercises — chapter 7

Chapter synthesis

The thesis is an argument whose every element points to evidence; the archive releases only what is safe and verified, failing closed; synthetic data are labelled as invented or calibrated; and limitations say which way they bias the result.

Continue studying — chapter 07 of 08

← Chapter 06 (draft) · Course chapter index · Chapter 08 (draft) →