◇ OpenCyber Academy
Doctoral research · Chapter 7 · English

Chapter 7 — Thesis synthesis and reproducibility archive

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 7 — Thesis synthesis and reproducibility archive

Lesson map

Lesson 07 published. Map, example, EX039.

flowchart TD
  N0["Learning outcomes"]
  N1["7.1 Synthesis"]
  N2["Lab — Complexity and excess"]
  N3["7.2 Reading the correlation"]
  N4["7.3 The archive"]
  N5["7.4 What can and cannot be reproduced"]
  N6["7.5 Sharing responsibly"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX039"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["7.1 Synthesis"]
  N2["Lab — Complexity and excess"]
  N3["7.2 Reading the correlation"]
  N4["7.3 The archive"]
  N5["7.4 What can and cannot be reproduced"]
  N6["7.5 Sharing responsibly"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX039"]

[Learning outcomes]
    |
    v
[7.1 Synthesis]
    |
    v
[Lab — Complexity and excess]
    |
    v
[7.2 Reading the correlation]
    |
    v
[7.3 The archive]
    |
    v
[7.4 What can and cannot be reproduced]
    |
    v
[7.5 Sharing responsibly]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX039]

Learning outcomes

You will (1) combine results into one argument; (2) assess the complexity relationship; (3) build a reproducibility archive that protects partners.

7.1 Synthesis

The synthesis chapter draws the studies together. Rhea's argument runs: excess privilege can be coded reliably; it is common in the partners' directories; units with more complex role models tend to have more of it; and a structured review removes substantially more of it than the usual review, at modest time cost and without a higher rate of wrongful removal. Each step is linked to evidence and its uncertainty. She states what is now known that was not before, and how confident readers should be, instead of repeating each chapter's results in turn.

Lab — Complexity and excess

"""Spearman correlation between role-model complexity and excess-privilege rate across synthetic business units (stdlib only)."""
units = [("U01", 3.1, 0.08), ("U02", 4.6, 0.12), ("U03", 2.2, 0.05), ("U04", 6.8, 0.21), ("U05", 5.0, 0.11),
         ("U06", 3.9, 0.14), ("U07", 7.5, 0.19), ("U08", 2.8, 0.07), ("U09", 5.9, 0.17), ("U10", 4.1, 0.09)]
def ranks(v):
    o = sorted(range(len(v)), key=lambda i: v[i]); r = [0]*len(v)
    for k, i in enumerate(o): r[i] = k + 1
    return r
x = ranks([u[1] for u in units]); y = ranks([u[2] for u in units]); n = len(units)
d2 = sum((p - q)**2 for p, q in zip(x, y))
print(f"units={n} sum_d2={d2} spearman_rho={1 - 6*d2/(n*(n*n-1)):.2f}")

Run with python3 lab_corr.py. Verified output:

units=10 sum_d2=16 spearman_rho=0.90

7.2 Reading the correlation

Across ten synthetic units, the Spearman correlation between complexity and excess rate is 0.90, a strong positive association. Rhea treats it carefully. Ten units give a wide interval. The within-partner check shows the pattern in two partners but weaker in the third, where a recent reorganisation created many temporary roles. And correlation cannot show that complexity causes excess; both may stem from fast change. She concludes that complexity is a useful warning sign for where to focus reviews, which is a practical claim the evidence supports, rather than a causal claim it does not.

7.3 The archive

The reproducibility archive contains the codebook with version history, analysis scripts, the synthetic datasets used throughout this book, the decision log, and a guide to rerunning each analysis. Scripts use only the Python standard library so they run without installation. A short readme maps each table and figure in the thesis to the script that produced it. Real partner exports are not included: they stay on the encrypted university server until the agreed deletion date. The archive lets others check methods and logic even where exact figures cannot be reproduced.

7.4 What can and cannot be reproduced

Rhea is explicit. Anyone can rerun the agreement, correlation and comparison analyses on the synthetic data and get the published synthetic figures. Nobody outside the project can reproduce the real figures, because partner data cannot be shared. For those, she publishes summary statistics, the code that produced them and synthetic data with the same structure, so the method can be scrutinised. Being clear about this boundary is more honest than either claiming full reproducibility or offering nothing, and it is what examiners and journals increasingly expect from studies using sensitive data.

7.5 Sharing responsibly

Before releasing anything, Rhea removes details that could identify a partner, such as system names, role names and unit sizes that would be recognisable. She checks that no combination of remaining fields reveals an organisation, and asks each partner to review the release. She shares the codebook and structured review procedure openly, because practitioners can use them directly. Balancing openness with the duty of care owed to the organisations that trusted her is part of research integrity, not an obstacle to it, and future partners are more likely to cooperate with researchers who handle data this way.

Worked example

The synthesis links reliable coding, common excess, a 0.90 complexity correlation treated as a warning sign, and a structured review that removes more excess safely; the archive supports reruns on synthetic data.

Synthetic case. Units, correlation and archive contents are synthetic.

Errors and limits

Presenting a strong correlation across few units as a causal finding overstates what the design can show.

Exercises

Synthesis

A careful synthesis claims only what the evidence supports, and a well-bounded archive lets others check the methods without exposing partners.

Continue studying — chapter 07 of 08

← Chapter 06 (draft) · Course chapter index · Chapter 08 (draft) →