◇ OpenCyber Academy
Doctoral research · Chapter 7 · English

Chapter 7 — Thesis synthesis and reproducibility archive

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 7 — Thesis synthesis and reproducibility archive

Lesson map

Lesson 07 published. Map, example, EX039.

flowchart TD
  N0["Learning outcomes"]
  N1["7.1 Synthesis"]
  N2["7.2 The archive"]
  N3["7.3 What can and cannot be reproduced"]
  N4["7.4 Sharing responsibly"]
  N5["Versioning and citation"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX039"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["7.1 Synthesis"]
  N2["7.2 The archive"]
  N3["7.3 What can and cannot be reproduced"]
  N4["7.4 Sharing responsibly"]
  N5["Versioning and citation"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX039"]

[Learning outcomes]
    |
    v
[7.1 Synthesis]
    |
    v
[7.2 The archive]
    |
    v
[7.3 What can and cannot be reproduced]
    |
    v
[7.4 Sharing responsibly]
    |
    v
[Versioning and citation]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX039]

Learning outcomes

You will (1) synthesise findings; (2) build a reproducibility archive; (3) share synthetic materials.

7.1 Synthesis

The thesis brings together three contributions: a modern replication of the complexity-error association, a validated codebook and an evaluated review procedure. Ines writes the synthesis chapter around what practitioners should do differently: measure rule-base hygiene periodically, use the procedure with owner confirmation, and expect larger rule bases to need more review effort. Each recommendation is tied to evidence and its limits.

The synthesis chapter draws the separate studies together into one argument, showing how each result supports or qualifies the main claim. Ines states what is now known that was not known before, and how confident readers should be, rather than repeating each result in turn.

7.2 The archive

Real partner configurations cannot be published, so Ines builds a synthetic archive that mirrors their structure: rule bases with the same size distribution and error proportions, generated by documented scripts. The archive includes the codebook, all analysis scripts, the decision log and instructions to reproduce every table. A colleague who has not seen the study follows the instructions and reproduces the main results from the synthetic data.

The archive holds the codebook, scripts, synthetic examples and a readme explaining how to rerun each analysis. Partner configurations themselves are excluded under the data agreement.

7.3 What can and cannot be reproduced

Ines is explicit that the synthetic archive reproduces the analysis, not the original findings, which depend on partner data. She describes how partners could rerun the scripts on their own configurations, and offers the procedure under an open licence so others can test it. This is the honest limit of reproducibility when data are sensitive.

Ines states plainly which analyses others can rerun from the archive and which depend on partner data that cannot be shared. For the latter she provides summary statistics and synthetic data with the same structure, so methods can still be checked, even though exact figures cannot be reproduced.

7.4 Sharing responsibly

Before release, she checks that synthetic rules cannot be mapped back to partners: pseudonymised ranges are replaced with documentation ranges, rule names are generic and size distributions are smoothed. The partners review the archive and approve publication.

Sharing research outputs must not expose the partners. Before releasing anything, Ines removes identifying details such as names, address ranges and rule comments, checks that no combination of remaining fields reveals an organisation, and has partners confirm the release. She shares methods, codebooks and synthetic data openly, while keeping real configurations restricted, balancing openness with the duty of care she owes the organisations that trusted her.

Versioning and citation

The archive is deposited in the university's repository with a persistent identifier and a version number, so later readers can cite exactly the version used in the thesis. When Ines corrects a script after publication, she releases a new version with a change note rather than overwriting the old one, because a reader trying to reproduce a published table needs the code that produced it. The thesis cites the archive version explicitly, and each table caption names the script that generated it, so the link between claim and code is visible on the page.

Worked example

The archive contains five folders: codebook, scripts, synthetic data, decision log and instructions; the colleague's reproduction matches every table to two decimal places.

Synthetic case. The archive is synthetic.

Errors and limits

Releasing partner-derived data without checks risks exposing network designs.

Exercises

Synthesis

A careful synthetic archive makes the analysis reproducible without exposing partners.

Continue studying — chapter 07 of 08

← Chapter 06 (draft) · Course chapter index · Chapter 08 (draft) →