◇ OpenCyber Academy
Doctoral research · Chapter 6 · English

Chapter 6 — Peer critique and revision

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 6 — Peer critique and revision

Lesson map

Lesson 06 published. Map, example, EX033.

flowchart TD
  N0["Learning outcomes"]
  N1["6.1 Internal seminar"]
  N2["6.2 Responding to reviewers"]
  N3["6.3 The observation question"]
  N4["6.4 Revising the procedure"]
  N5["6.5 Keeping a decision log"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX033"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["6.1 Internal seminar"]
  N2["6.2 Responding to reviewers"]
  N3["6.3 The observation question"]
  N4["6.4 Revising the procedure"]
  N5["6.5 Keeping a decision log"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX033"]

[Learning outcomes]
    |
    v
[6.1 Internal seminar]
    |
    v
[6.2 Responding to reviewers]
    |
    v
[6.3 The observation question]
    |
    v
[6.4 Revising the procedure]
    |
    v
[6.5 Keeping a decision log]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX033]

Learning outcomes

You will (1) present the study for critique; (2) respond to reviewers with evidence; (3) keep a decision log.

6.1 Internal seminar

Rhea presents her design and results at a department seminar before submitting a paper. She shows the codebook, agreement figures, randomisation, validation and limitations, then invites the hardest questions colleagues can find. One asks whether reviewers in the structured condition worked harder simply because they knew they were observed. Another asks whether usage logs miss access through shared service accounts. Rhea records every question she could not answer well, because those questions show exactly where the thesis needs clearer explanation, further analysis or a stated limitation.

6.2 Responding to reviewers

A workshop paper based on chapter 5 returns with three reviews. Rhea answers each comment in a response table: the comment, her reply, the evidence and the page where the change appears. Where a reviewer is right, she changes the paper. Where she disagrees, she explains why with evidence rather than assertion. One reviewer asks for a sensitivity analysis excluding the partner with the most units; Rhea runs it and reports that the direction of the effect holds with a wider interval. Specific, polite and evidenced replies earn more trust than defensive ones.

6.3 The observation question

The concern about observation is fair. Rhea observed only two sittings per unit, in both conditions, so attention was balanced. She also compares removal rates in observed and unobserved sittings within the structured condition and finds them similar. That does not rule out every effect of knowing about the study, since reviewers knew their unit was participating, but it reduces the worry. She adds the analysis to the thesis and states the remaining limitation plainly, rather than claiming the issue is resolved. Examiners value seeing a concern taken seriously and partially tested.

6.4 Revising the procedure

Feedback also improves the structured review itself. Reviewers asked for entitlements to be grouped by system, and for clearer marking of rarely used access that is still needed, such as emergency roles. Rhea revises the procedure for any future use, recording each change with its reason. She does not apply the revision retrospectively to her results: the study tested version 2 of the procedure, and the thesis says so. Keeping tested and improved versions separate prevents readers from assuming the revised procedure has the same evidence behind it as the one actually trialled.

6.5 Keeping a decision log

Throughout, Rhea keeps a decision log: each significant choice with date, options, decision and reason. It records why ninety days was chosen for dormancy, why units rather than reviewers were randomised and why one partner's legacy system was marked unknown. Over several years memory fades and examiners ask why. The log answers with evidence, protects against unconsciously rewriting the history of the project and helps any future researcher who wants to replicate or extend the work. It also makes writing the methods chapter far quicker.

Worked example

Seminar and reviewer critique lead to a sensitivity analysis, an observed-versus-unobserved comparison and a revised procedure version, all recorded in the decision log.

Synthetic case. Seminar, reviewers and analyses are synthetic.

Errors and limits

Applying procedure improvements retrospectively to reported results misrepresents what was actually tested.

Exercises

Synthesis

Critique handled with evidence, separate procedure versions and a decision log make the thesis stronger and more trustworthy.

Continue studying — chapter 06 of 08

← Chapter 05 (draft) · Course chapter index · Chapter 07 (draft) →