◇ OpenCyber Academy
Master's · Chapter 7 · English

Chapter 7 — Dissertation writing and limitations

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 7 — Dissertation writing and limitations

Lesson map

Lesson 07 published. Map, example, EX039.

flowchart TD
  N0["Learning outcomes"]
  N1["7.1 Structure"]
  N2["7.2 Findings versus interpretation"]
  N3["7.3 Limitations by validity type"]
  N4["7.4 Writing about attacks responsibly"]
  N5["7.5 Artefact appendix"]
  N6["7.7 Practice activities"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX039"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["7.1 Structure"]
  N2["7.2 Findings versus interpretation"]
  N3["7.3 Limitations by validity type"]
  N4["7.4 Writing about attacks responsibly"]
  N5["7.5 Artefact appendix"]
  N6["7.7 Practice activities"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX039"]

[Learning outcomes]
    |
    v
[7.1 Structure]
    |
    v
[7.2 Findings versus interpretation]
    |
    v
[7.3 Limitations by validity type]
    |
    v
[7.4 Writing about attacks responsibly]
    |
    v
[7.5 Artefact appendix]
    |
    v
[7.7 Practice activities]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX039]

Learning outcomes

You will (1) structure a dissertation so every chapter serves the research question; (2) write results and discussion that separate findings from interpretation; (3) write a limitations section organised by validity type; (4) handle security-sensitive details responsibly in writing; and (5) produce an artefact appendix that supports reproduction.

7.1 Structure

A common structure: introduction (problem, question, contribution summary); background and related work (from your matrix); method (design, data, attacks, metrics, analysis plan); results (primary, secondary, exploratory clearly labelled); discussion (interpretation, competing explanations, implications); limitations; conclusion; references; appendices (artefacts, extra tables). Write the method so a peer could repeat it; write the results so a reader can find the answer to the research question in one table. Leo's dissertation answers its question in one table of TPR at 1% FPR and macro-F1 for the four conditions.

7.2 Findings versus interpretation

Results state what was measured: "near deduplication reduced TPR at 1% FPR from 9.8% to 4.2%". Discussion interprets: "this suggests duplicated records drive much of the leakage, consistent with work on memorisation in language models". Keep them apart. In discussion, return to competing explanations and say which your ablations ruled out. Leo moved a recommendation that insurers should deduplicate from results to discussion and softened it. Read each results sentence and ask whether it reports a measurement; if it explains or recommends, move it.

7.3 Limitations by validity type

Organise limitations by the four validity types. Internal: could something other than the treatment explain the effect? (e.g., synthetic generator artefacts.) Construct: do the metrics measure the property? (e.g., TPR at 1% FPR under one attack family captures only some privacy risk.) External: does the result transfer? (e.g., one model architecture, synthetic notes, English only.) Conclusion: are the statistics adequate? (e.g., five seeds.) For each, say what you did and what remains. Limitations written this way are a strength; examiners distrust dissertations that claim none.

7.4 Writing about attacks responsibly

Describe attacks at the level needed for understanding and reproduction in research settings, cite the original papers, and avoid publishing operational detail that targets named live systems. Never include recovered personal data. If you found a vulnerability in a real product, follow coordinated disclosure before the dissertation is published. Label synthetic data and organisations as fiction throughout. Leo's dissertation describes the calibrated attack by citing its source and his configuration, without any recovered records. Ask your supervisor to read the attack description for operational detail before submission.

7.5 Artefact appendix

Include repository URL and commit, environment, commands to reproduce each table, data card for the synthetic generator, model card for any released model (Mitchell et al., arXiv:1810.03993), and hashes of key files. State what is not released and why. Leo's appendix includes the synthetic generator's data card, the repository commit, commands for each table and file hashes. He states that no model is released and why. Ask a peer to reproduce one table from the appendix alone before submission. Keep it current.

7.6 Worked example (synthetic Harbourline case)

Leo's limitations section has four short subsections. External validity: "We studied one transformer classifier fine-tuned on synthetic English notes; results may differ for generative models, other languages and real claims data." Construct validity: "We measured membership inference with two attacks; attribute inference and extraction were not studied." He moved a sentence about "insurers should deduplicate" out of results into discussion and softened it to "deduplication is a low-cost candidate control that insurers could evaluate". The extraction lab is cited in his background chapter as a simple illustration of why API output precision matters when describing threat models.

"""Lab X: model extraction of a toy linear scoring API by querying it.
The 'API' returns a real-valued score; d+1 well-chosen queries recover the
weights exactly, and rounding the output limits fidelity. Synthetic model."""
import random
rng = random.Random(9)
d = 5
secret_w = [round(rng.uniform(-2, 2), 3) for _ in range(d)]; secret_b = 0.42
def api(x, decimals=None):
    s = sum(a*b for a, b in zip(secret_w, x)) + secret_b
    return round(s, decimals) if decimals is not None else s
for dec in (None, 2, 0):
    b = api([0]*d, dec)
    w = [api([1 if j == i else 0 for j in range(d)], dec) - b for i in range(d)]
    err = max(abs(a-c) for a, c in zip(w, secret_w))
    test = [[rng.uniform(-1, 1) for _ in range(d)] for _ in range(500)]
    agree = sum((sum(a*c for a, c in zip(w, x))+b > 0) == (api(x) > 0) for x in test)/500
    print(f"output_rounding={str(dec):4s} queries={d+1} max_weight_error={err:.3f} label_agreement={agree:.3f}")

Expected output:

output_rounding=None queries=6 max_weight_error=0.000 label_agreement=1.000
output_rounding=2    queries=6 max_weight_error=0.004 label_agreement=0.998
output_rounding=0    queries=6 max_weight_error=0.534 label_agreement=0.882

7.7 Practice activities

  1. Write your limitations section by validity type. 2. Mark every sentence in your results chapter as finding or interpretation and move misplaced ones. 3. Draft the artefact appendix. For the limitations, use the four validity types and say for each what you did and what remains, as Leo did for one classifier and synthetic notes. For the results chapter, mark each sentence and move interpretations to the discussion. For the appendix, list repository, commit, environment, commands, cards and hashes, and state what is not released.

7.8 Errors and limits

Four errors recur, and each makes claims harder to trust; check your draft for all four:

Exercises

Synthesis

A dissertation is judged by how clearly its claims match its evidence. Separating findings from interpretation, writing specific limitations and handling attack details responsibly make the work credible and safe.

Continue studying — chapter 07 of 08

← Chapter 06 (draft) · Course chapter index · Chapter 08 (draft) →