OpenCyber Academy
Doctoral research · Chapter 6 · English
Course code: OCA-204Chapter 06: OCA-204-C006

Chapter 6 — Peer critique and revision

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 06 published. Map, example, EX033.

flowchart TD
  N0["Learning outcomes"]
  N1["6.1 How to review an AI-security paper"]
  N2["6.2 Writing critique that helps"]
  N3["6.3 Receiving critique"]
  N4["6.4 Revising claims to fit evidence"]
  N5["6.6 Practice activities"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX033"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["6.1 How to review an AI-security paper"]
  N2["6.2 Writing critique that helps"]
  N3["6.3 Receiving critique"]
  N4["6.4 Revising claims to fit evidence"]
  N5["6.6 Practice activities"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX033"]

[Learning outcomes]
    |
    v
[6.1 How to review an AI-security paper]
    |
    v
[6.2 Writing critique that helps]
    |
    v
[6.3 Receiving critique]
    |
    v
[6.4 Revising claims to fit evidence]
    |
    v
[6.6 Practice activities]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX033]

Learning outcomes

You will (1) review an AI-security paper against a structured checklist; (2) write critique that is specific, actionable and fair; (3) receive critique and separate valid objections from misunderstanding; (4) revise claims, analyses and text so that the evidence and the words match; and (5) write a response letter.

6.1 How to review an AI-security paper

Read in three passes. First, the claim: what exactly does the paper assert, and under which threat model? Second, the evidence: do the experiments match the claim's quantifiers (adversary, budget, models, tasks)? Third, the artefacts: are code, data and configurations available to check? A practical checklist, built from arXiv:1902.06705 and arXiv:2010.09470, asks: Is the threat model explicit? Are adaptive attacks included? Are baselines strong and tuned equally? Are metrics appropriate to base rates? Are intervals reported across seeds? Is there leakage between development and test data? Are utility costs measured? Are ethical issues and disclosure handled? Are limitations stated where a reader will see them?

6.2 Writing critique that helps

Good reviews summarise the paper fairly in a few sentences first, so the authors see you understood it. Then list major issues (that would change the conclusion if unresolved) separately from minor ones. For each major issue, say what is wrong, why it matters, and what evidence would resolve it: "The defence is evaluated only against the 120 payloads from prior work; since the defence modifies delimiters, please evaluate delimiter-forging payloads and report ASR as a function of attempts." Avoid tone that attacks the authors. Avoid asking for experiments that would not change your decision.

6.3 Receiving critique

Sort every comment into four bins: correct and fixable (do it), correct but out of scope (acknowledge and add a limitation), misunderstanding (the text was unclear — rewrite it) and disagreement (respond with evidence, respectfully). The third bin is larger than authors expect; if a careful reviewer misread you, other readers will too. A response letter quotes each comment and states what changed and where. Ana's response to the request for the payload corpus fits the out-of-scope bin: she acknowledged it and explained the gated release. Her retitling fits the first bin. Keep the sorted list with the response letter.

6.4 Revising claims to fit evidence

Revision usually narrows claims. Typical edits: replace "robust" with "reduced ASR under the evaluated threat model"; replace "generalises" with "held on two of three models"; move a limitation from the appendix to the abstract; add a figure of ASR versus attack budget; report the red team's best attack. Claims that shrink but survive are stronger than claims that are broad and fragile. Ana's retitling from "prevents" to "reduces" and her report of a smaller effect on the third model are examples. Check each claim against the table that supports it after every revision.

6.5 Worked example (synthetic MAAL case)

MAAL's internal review of Ana's draft raised three major issues. (1) "Separation prevents injection" in the title contradicted the 5.8–8.9% residual ASR; she retitled it to "reduces". (2) Both models were from one family; a reviewer asked whether the effect transfers. Ana added a third open model from a different family as a validation-only run: the reduction held in direction but was smaller (3.1 points, CI 1.4–4.9), and she reported it. (3) Utility was measured with an LLM judge. Ana added a 200-task human audit with two annotators, obtaining Cohen's κ = 0.81, and reported judge–human agreement. A minor comment asked for the full payload corpus; she explained the gated release plan from Chapter 3 instead.

6.6 Practice activities

  1. Review a published paper in your area with the checklist and write a one-page review. 2. Exchange drafts with a peer and write response letters to each other. 3. List every claim in your abstract and the table or figure that supports it. For the review, use the checklist questions and separate major from minor issues, with what evidence would resolve each major one. For the response letters, quote each comment, sort it into the four bins and state what changed and where. For the claims list, any claim without a supporting table or figure should be removed or narrowed, as Ana's title was.

6.7 Errors and limits

Each of these errors wastes the review:

Exercises

Synthesis

Critique is a validity instrument. Reviewing others teaches you what evidence persuades; revising your own work aligns the words with that evidence, which usually means narrower and sturdier claims.

Continue studying — chapter 06 of 08

← Chapter 05 (draft) · Course chapter index · Chapter 07 (draft) →

Worked answers

EX033. Answers vary; a full answer cites sections for threat model, adaptive attacks, baselines, metrics and intervals.

EX034. "What: evaluation uses only 120 static payloads. Why: the defence changes delimiters, so attackers will target them; static payloads cannot measure that. Evidence: evaluate delimiter-forging and echo payloads with a stated budget and report ASR versus attempts."

EX035. Misunderstanding (or disagreement). Respond: "RQ4 compares separation with and without allowlisting; Table 5 shows a 5.2-point reduction (CI 3.9–6.6) over allowlist alone. We have clarified this in the introduction."

EX036. "Under our threat model, adding instruction–data separation to tool allowlisting reduced unauthorised tool calls by 5.2 points on two models (residual ASR 5.8%, up to 8.9% under independent red-teaming); it does not prevent injection."

EX037. κ = 0.81 means strong agreement beyond chance between the two humans. It does not show the labels are correct, nor that the LLM judge is valid unless judge labels are compared with the human labels.

EX038. "Reviewer 2: 'Does the effect transfer beyond one model family?' We added model C from a different family as a validation-only run (Section 5.4, Table 6). The reduction held in direction but was smaller: 3.1 points (95% CI 1.4–4.9). We now state this in the abstract."