◇ OpenCyber Academy
Master's · Chapter 8 · English

Chapter 8 — Independent review and defense

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 8 — Independent review and defense

Lesson map

Lesson 08 published. Map, example, EX050.

flowchart TD
  N0["Learning outcomes"]
  N1["8.1 Independent methods review"]
  N2["8.2 Likely viva questions"]
  N3["8.3 Responding to findings"]
  N4["8.4 Returning value"]
  N5["8.5 After the defence"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX050"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["8.1 Independent methods review"]
  N2["8.2 Likely viva questions"]
  N3["8.3 Responding to findings"]
  N4["8.4 Returning value"]
  N5["8.5 After the defence"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX050"]

[Learning outcomes]
    |
    v
[8.1 Independent methods review]
    |
    v
[8.2 Likely viva questions]
    |
    v
[8.3 Responding to findings]
    |
    v
[8.4 Returning value]
    |
    v
[8.5 After the defence]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX050]

Learning outcomes

You will (1) commission an independent methods review before submission; (2) prepare for viva questions on design and ethics; (3) respond to review findings transparently; (4) plan dissemination back to the community that provided data.

8.1 Independent methods review

A colleague outside the project reruns the analysis from the archive using synthetic sample files and checks that every figure matches. A second reader checks that no restricted information leaked into text, tables or figure captions. Both reviews are documented.

In Lucía's case, the figure-checking colleague found that all medians reproduced exactly from the archive. The leak reviewer found one caption that named a port's city. Each finding and its correction went into the change log, so the examiners could see both reviews were real, not formal steps.

8.2 Likely viva questions

Why not a randomised design? How did you handle confounding? What did members consent to? What would change your conclusion? Why medians? Is the STIX validation rule set complete? Practise answers that cite the dissertation's own pages.

For Lucía, the confounding answer cites the matched comparison and the before-and-after stratum; the medians answer cites the skewed timing data; and "what would change your conclusion" cites evidence that the before-and-after stratum shows no drop on joining. Answers rooted in her own design are harder to dislodge.

8.3 Responding to findings

If the reviewer finds a figure that does not match, fix it, record the correction in a change log, and check whether any conclusion changes. Examiners respect a visible correction more than an invisible one discovered later.

Lucía's change log shows the pattern for a different finding, the caption that named a port's city: the change, its date and the board notification are all recorded. Had a median failed to reproduce, the same log would show the corrected value and whether the 7.0-hour gap still held.

8.4 Returning value

CPSG gave data; the community should get results first. Lucía presents the brief to the governance board before public defence, under the agreed aggregation rules. NIST SP 800-150 stresses that sharing relationships rely on trust and reciprocity; research that extracts without returning erodes both.

The brief also carries the data-quality note, which members can act on immediately: by fixing exports that lack valid_from or spec_version, producers make the next study, and their own consumers' automation, more reliable. The 3.0- and 10.0-hour medians reached the CPSG board before the examiners saw them, which keeps the reciprocity NIST SP 800-150 describes.

8.5 After the defence

Delete raw timings as agreed, archive code and synthetic samples, and record the deletion. If publishing a paper, recheck that the published version still meets the board's conditions.

For Lucía, the deletion record is the last item the board receives: which files were deleted, when, and by whom. Any paper drafted later from the dissertation is rechecked against the three board conditions before submission. Archived code and synthetic samples stay available, so the study can still be rerun after the raw timings are gone; the deletion does not weaken reproducibility, because the archive was designed around synthetic files from the start.

Worked example

The leak reviewer finds that one figure caption names a port's city, which would identify the member. Lucía replaces it with "Member C", logs the change, and notifies the board. The figure-checking colleague finds all medians reproduce exactly.

Synthetic case. The review findings and board notification are invented.

Errors and limits

A friend who helped with analysis is not independent. Presenting to the community after publication may breach the agreement.

Exercises

Synthesis

Get independent eyes on numbers and leaks, rehearse design and ethics answers, correct in the open, and give results back to the people who shared.

Continue studying — chapter 08 of 08

← Chapter 07 (draft) · Course chapter index · Worked results ↓ · Next course via chapter index →