Chapter 7 — Thesis synthesis and reproducibility archive
Lesson map
Lesson 07 published. Map, example, EX039.
flowchart TD
N0["Learning outcomes"]
N1["7.1 Thesis structure"]
N2["7.2 The archive"]
N3["Lab — manifest verification"]
N4["7.3 Reading the result"]
N5["7.4 Why hash manifests"]
N6["7.5 Images or generator"]
N7["7.6 Licences"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX039"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["7.1 Thesis structure"]
N2["7.2 The archive"]
N3["Lab — manifest verification"]
N4["7.3 Reading the result"]
N5["7.4 Why hash manifests"]
N6["7.5 Images or generator"]
N7["7.6 Licences"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX039"]
[Learning outcomes]
|
v
[7.1 Thesis structure]
|
v
[7.2 The archive]
|
v
[Lab — manifest verification]
|
v
[7.3 Reading the result]
|
v
[7.4 Why hash manifests]
|
v
[7.5 Images or generator]
|
v
[7.6 Licences]
|
v
[Worked example] --> [Exercise EX039]
Learning outcomes
You will (1) structure a doctoral thesis in digital forensics; (2) build a reproducibility archive; (3) verify archive integrity with hashes; (4) document what others need to rerun the work.
7.1 Thesis structure
Introduction; background (file systems per Carrier, carving, validation standards); generator design; pilot; main study; validation by a second lab; discussion; limitations; conclusion; appendices with manifests and logs. The order mirrors the reasoning of the study: the generator must be described before the pilot can be judged, and the pilot before the main study. The limitations chapter lists the synthetic nature of the images, the single file family and the single method run in the main study. Appendices let an examiner check any figure against the raw logs without contacting Ines.
7.2 The archive
Data (images or the generator plus seeds), code (generator, carvers, scorer), environment (pinned versions), results, and a README describing how to reproduce each table. Each result file has a SHA-256 entry in a manifest at the top of the archive. The README lists commands in order and the expected output of each, so a reader can tell immediately whether their run reproduced the table or diverged. Ines also lists the archive's own size and the time a full run takes on an ordinary laptop.
Lab — manifest verification
"""Lab (ch 07): SHA-256 manifest for a reproducibility archive, then verify a copy with one altered file."""
import hashlib
orig = {"data/img-contig.bin": b"synthetic-image-1", "data/img-frag10.bin": b"synthetic-image-2",
"code/carve.py": b"print('carve')", "results/recall.csv": b"img,recall\ncontig,0.96\n"}
manifest = {p: hashlib.sha256(b).hexdigest() for p, b in orig.items()}
copy = dict(orig); copy["results/recall.csv"] = b"img,recall\ncontig,0.98\n"
for p in sorted(manifest):
ok = hashlib.sha256(copy[p]).hexdigest() == manifest[p]
print(p, "OK" if ok else "MISMATCH")
print("archive verified" if all(hashlib.sha256(copy[p]).hexdigest() == h for p, h in manifest.items()) else "archive FAILED verification")
Run with python3 lab_manifest.py. Verified output:
code/carve.py OK
data/img-contig.bin OK
data/img-frag10.bin OK
results/recall.csv MISMATCH
archive FAILED verification
7.3 Reading the result
Three of four files in the copied archive match their SHA-256 manifest entries. results/recall.csv does not: its recall value changed from 0.96 to 0.98. Verification fails, exactly as it should when any result is altered after publication. The lab therefore demonstrates the check on purpose: a single changed digit is enough to break the manifest. Ines runs the same verification on her real archive before submission, and the thesis states that all entries matched at that date. Because the failure names results/recall.csv, Ines knows which table to regenerate.
7.4 Why hash manifests
The same principle protects evidence: ISO/IEC 27037 stresses preserving integrity from acquisition onward, commonly demonstrated by recording hash values. Applying it to research archives lets examiners trust that published results are those the thesis reports. The manifest itself is part of the thesis appendix, so altering both a result and its hash would require changing the deposited thesis as well. Ines also records the date the manifest was produced and the tag of the code that produced the results it lists.
7.5 Images or generator
Publishing the generator and seeds is smaller than publishing images and lets others regenerate them exactly; Ines also publishes one sample image for quick checks. The sample image lets a reader check the carver quickly without running the generator, and its hash is in the manifest. If a regenerated image ever differs from the published sample, the difference shows that some part of the environment has changed, as happened with the Python version during the second laboratory's rerun. A reader regenerating all 150 target files needs only the seed list and the generator.
7.6 Licences
Code under an open licence; text under the university's terms; no third-party data with restrictive terms. Ines checked each dependency's licence before release. Because her labs use only the Python standard library, no external package licence applies to the code. The README states the licence of each part of the archive, so a laboratory can reuse the generator in its own validation work without asking permission, and the thesis cites the licence text with a link. The university's repository office confirmed that depositing the archive does not conflict with its terms.
7.7 Reproduction test
A colleague follows the README on a clean machine and regenerates every table before submission. The colleague has not seen the code before and is asked not to contact Ines unless blocked. Each point where they had to guess is treated as a fault in the README and fixed, and the test is repeated until it passes with no questions. Their report, with the date and the tag of the archive tested, is filed with the thesis. The laptop-path error found during this test is listed as an example of why the step is worth its time.
Logs and notebooks
Analysis logs record every run with date, seed, code version and output hash. Examiners can trace any figure in the thesis to the exact run that produced it.
Worked example
The reproduction test fails on one table because a script read a file path specific to Ines's laptop. She replaces it with a relative path, reruns, updates the manifest and records the fix.
Synthetic case. The archive is invented.
Errors and limits
A hash proves integrity, not correctness; a wrong result can be faithfully preserved.
Exercises
- EX039 — Manifest failure. Which archive file failed verification, and why? Rubric: 3 = results/recall.csv; value changed 0.96→0.98; 2 = file only; 1 = wrong; 0 = none.
- EX040 — Archive contents. List five components of the archive. Rubric: 3 = data/generator+seeds, code, environment, results, README; 2 = three; 1 = one; 0 = none.
- EX041 — Evidence parallel. How does the manifest relate to evidence handling? Rubric: 3 = integrity via hashes, as in ISO/IEC 27037 preservation; 2 = partial; 1 = vague; 0 = none.
- EX042 — Generator over images. Why publish the generator and seeds rather than only images? Rubric: 3 = smaller; exact regeneration; 2 = one; 1 = no reason; 0 = none.
- EX043 — Path bug. What broke the reproduction test, and how was it fixed? Rubric: 3 = absolute laptop path; relative path, manifest updated; 2 = one; 1 = vague; 0 = none.
- EX044 — Integrity vs correctness. What does a matching hash not prove? Rubric: 3 = that results are correct; 2 = partial; 1 = proves all; 0 = none.
Synthesis
Structure the thesis clearly, archive everything needed to rerun it, verify with hashes and test reproduction on a clean machine.
Continue studying — chapter 07 of 08
← Chapter 06 (draft) · Course chapter index · Chapter 08 (draft) →