◇ OpenCyber Academy
Doctoral research · Chapter 7 · English

Chapter 7 — Thesis synthesis and reproducibility archive

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 7 — Thesis synthesis and reproducibility archive

Lesson map

Lesson 07 published. Map, example, EX039.

flowchart TD
  N0["Learning outcomes"]
  N1["7.1 The thesis argument"]
  N2["7.2 The reproducibility archive"]
  N3["7.3 Checking FAIR properties"]
  N4["7.4 Improving the archive"]
  N5["Lab — FAIR checklist scores for dataset records"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5
  N5 --> EX["Worked example"]
  EX --> Q["Exercise EX039"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["7.1 The thesis argument"]
  N2["7.2 The reproducibility archive"]
  N3["7.3 Checking FAIR properties"]
  N4["7.4 Improving the archive"]
  N5["Lab — FAIR checklist scores for dataset records"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX039"]

[Learning outcomes]
    |
    v
[7.1 The thesis argument]
    |
    v
[7.2 The reproducibility archive]
    |
    v
[7.3 Checking FAIR properties]
    |
    v
[7.4 Improving the archive]
    |
    v
[Lab — FAIR checklist scores for dataset records]
    |
    v
[Worked example] --> [Exercise EX039]

Learning outcomes

You will (1) synthesise findings into a coherent thesis; (2) build a reproducibility archive; (3) assess the archive against FAIR principles.

7.1 The thesis argument

Nadia's thesis argues that, in the setting studied, governance decisions about linkage and quality rules shape official statistics as much as data collection does, and that the most influential decisions received the least governance attention. Each chapter supports one step: the decision matrix, the baseline replication, the main simulation, the validation and the critique response. She writes the argument on one page before revising chapters, so that every section contributes to the claim and nothing is included simply because it was work she did.

7.2 The reproducibility archive

Nadia builds an archive containing the replica generator, analysis scripts, fixed seeds, expected outputs, the decision log, interview protocol and anonymised coding frame. Confidential material, including any Westmark internal documents, is excluded and described instead. Wilkinson and colleagues' FAIR principles state that data and metadata should be findable, accessible, interoperable and reusable, with persistent identifiers, rich metadata, standard vocabularies, clear licences and provenance. She treats the principles as design goals for the archive rather than as a certificate it either has or lacks.

7.3 Checking FAIR properties

To test her archive against those goals, Nadia scores four synthetic dataset records using an eight-item checklist. The lab reports that the business register record meets 8 of 8 items, the linked employment record 4 of 8, the trade survey 4 of 8 and the pilot extract 1 of 8, a mean score of 0.53. The linked employment record lacks a persistent identifier, search index entry, standard vocabulary and licence, which would make reuse difficult even for colleagues inside Westmark. The pilot extract meets only the open protocol item, which is typical of files produced quickly for a test and never described afterwards. Such files are the first to become unusable.

7.4 Improving the archive

Nadia applies the lessons to her own archive. She obtains a persistent identifier through the university repository, writes metadata describing variables with standard codes, chooses an open licence for code and synthetic data and documents provenance from generator to figure. Access conditions state that confidential microdata are not included and how researchers can apply to Westmark separately. Prof. Eriksen tests the archive by asking a new student to regenerate two figures; the student succeeds without help in an afternoon. No step required her help.

Lab — FAIR checklist scores for dataset records

"""Score synthetic dataset metadata records against a simple FAIR checklist (stdlib only)."""
checks = ["persistent_id", "rich_metadata", "searchable_index", "open_protocol", "access_conditions",
          "standard_vocabulary", "licence", "provenance"]
sets = {
    "business_register_2025": {"persistent_id", "rich_metadata", "searchable_index", "open_protocol", "licence", "provenance", "standard_vocabulary", "access_conditions"},
    "linked_employment_2024": {"rich_metadata", "open_protocol", "access_conditions", "provenance"},
    "trade_survey_2023":      {"persistent_id", "searchable_index", "open_protocol", "licence"},
    "pilot_linkage_extract":  {"open_protocol"},
}
for name, have in sets.items():
    miss = [c for c in checks if c not in have]
    print(f"{name:24s} {len(have)}/{len(checks)}  missing: {', '.join(miss) if miss else '-'}")
avg = sum(len(h) for h in sets.values()) / (len(sets) * len(checks))
print(f"mean checklist score {avg:.2f}")

Run with python3 lab_fair.py. Verified output:

business_register_2025   8/8  missing: -
linked_employment_2024   4/8  missing: persistent_id, searchable_index, standard_vocabulary, licence
trade_survey_2023        4/8  missing: rich_metadata, access_conditions, standard_vocabulary, provenance
pilot_linkage_extract    1/8  missing: persistent_id, rich_metadata, searchable_index, access_conditions, standard_vocabulary, licence, provenance
mean checklist score 0.53

Worked example

Nadia writes a one-page thesis argument, builds an archive excluding confidential material, scores four synthetic records from 8 of 8 to 1 of 8 (mean 0.53) and improves her archive until a new student regenerates figures unaided.

Synthetic case. Dataset records are synthetic; Wilkinson et al. (2016) is real.

Errors and limits

A checklist simplifies FAIR principles; some items need judgement, not ticks.

Exercises

Continue studying — lesson 07 (chapter 07 of 08)

← Chapter 06 · Course chapter index · Chapter 08 →

Continue to the next authored lesson.