Chapter 7 — Thesis synthesis and reproducibility archive
Lesson map
Lesson 07 published. Map, example, EX039.
flowchart TD
N0["Learning outcomes"]
N1["7.1 Synthesis"]
N2["7.2 The archive"]
N3["Lab — Manifest digests for a reproducibility archive"]
N4["7.3 Protecting participants"]
N5["7.4 What the archive enables"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX039"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["7.1 Synthesis"]
N2["7.2 The archive"]
N3["Lab — Manifest digests for a reproducibility archive"]
N4["7.3 Protecting participants"]
N5["7.4 What the archive enables"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX039"]
[Learning outcomes]
|
v
[7.1 Synthesis]
|
v
[7.2 The archive]
|
v
[Lab — Manifest digests for a reproducibility archive]
|
v
[7.3 Protecting participants]
|
v
[7.4 What the archive enables]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX039]
Learning outcomes
You will (1) synthesise findings into a coherent thesis; (2) build a reproducibility archive; (3) protect participants in shared data.
7.1 Synthesis
Mireille's thesis connects four parts: the critique of click rates, the measurement protocol, the stepped study and the replication. She frames the overall contribution as evidence that reporting-focused programmes can improve timely reporting when reporting is cheap and feedback is fast, consistent with the compliance budget idea of Beautement, Sasse and Wonham (2008). She also lists what remains unknown: which component matters most, whether effects last beyond fourteen months and whether reporting translates into fewer harmful incidents. Each unknown is linked to a concrete design that could answer it.
7.2 The archive
Mireille builds an archive: pre-registration, analysis code, the coding guide, the change log, anonymised department-month data, the simulation templates with difficulty ratings and the ethics approvals. Each file is listed in a manifest with a SHA-256 digest, so anyone can check that files have not changed since publication. The lab builds a manifest for a small synthetic archive, then edits one file and shows that the check reports a mismatch. She tests the archive by asking a colleague to rerun the main analysis from it alone on a fresh computer, which succeeds after one missing file is added.
Lab — Manifest digests for a reproducibility archive
# Build a SHA-256 manifest for a synthetic archive, then detect an edited file.
import hashlib
files = {"prereg.md": b"H1: reporting rises after switch\n", "dept_month.csv": b"dept,month,rate\nG1,1,0.24\n",
"coding_guide.md": b"Suspicious if sender spoofed or link mismatched\n"}
manifest = {n: hashlib.sha256(b).hexdigest() for n, b in files.items()}
files["dept_month.csv"] = b"dept,month,rate\nG1,1,0.42\n"
for n, b in files.items():
print(f"{n}: {'OK' if hashlib.sha256(b).hexdigest() == manifest[n] else 'MISMATCH'} {manifest[n][:12]}")
Run with python3 lab_archive.py. Verified output:
prereg.md: OK 32460485b7ca
dept_month.csv: MISMATCH 92dd85d36614
coding_guide.md: OK 0b96e40b1bab
7.3 Protecting participants
Even department-level data could identify people in small departments. Mireille merges departments with fewer than twenty staff into larger groups before release and removes dates finer than a month. Simulation templates are shared without Vantor's branding. She checks that no free-text fields remain. Vantor's data protection officer reviews the archive before release. Reproducibility and privacy can conflict, and she documents each choice so others understand what was withheld and why. The data protection officer signs a short statement confirming the review, which is included in the archive, and Mireille records which variables were removed or coarsened.
7.4 What the archive enables
With the archive, another researcher can rerun every analysis, check coding decisions and reuse the protocol in a new organisation. Prof. Aalto notes that this matters especially in behaviour research, where results often fail to replicate and organisations change quickly. Mireille deposits the archive in Calder's repository with a persistent identifier and a licence that permits reuse. She also writes a short practitioner guide so organisations can apply the protocol without reading the full thesis. She plans to update the archive if errors are found.
Worked example
Mireille synthesises her findings with clear unknowns, builds a digest-checked archive, protects participants through aggregation and deposits it with a practitioner guide.
Synthetic case. Archive contents are synthetic; Beautement, Sasse and Wonham (2008) is real.
Errors and limits
Digests prove files are unchanged, not that they are correct. Aggregation reduces analytical detail.
Exercises
- EX039 — Overall contribution. How does Mireille frame her overall contribution? Rubric: 3 = reporting-focused programmes improve timely reporting when reporting is cheap and feedback fast;; 2 = conditional;; 1 = universal;; 0 = none.
- EX040 — Open unknowns. Name one question the thesis leaves open. Rubric: 3 = which component matters, whether effects last, whether incidents fall;; 2 = one;; 1 = vague;; 0 = none.
- EX041 — Manifest check. What did the archive lab report after editing one file? Rubric: 3 = MISMATCH for the edited CSV;; 2 = mismatch;; 1 = vague;; 0 = none.
- EX042 — Small departments. How were small departments protected? Rubric: 3 = merged if under twenty staff;; 2 = merged;; 1 = vague;; 0 = none.
- EX043 — Privacy review. Who reviewed the archive before release? Rubric: 3 = Vantor's data protection officer;; 2 = DPO;; 1 = vague;; 0 = none.
- EX044 — Practitioner guide. Why write a short practitioner guide? Rubric: 3 = so organisations can apply the protocol without the full thesis;; 2 = uptake;; 1 = vague;; 0 = none.
Synthesis
A thesis synthesis states contributions and unknowns, and a privacy-aware archive lets others verify and reuse the work.
Continue studying — chapter 07 of 08
← Chapter 06 (draft) · Course chapter index · Chapter 08 (draft) →