Chapter 6 — Peer critique and revision
Lesson map
Lesson 06 published. Map, example, EX033.
flowchart TD
N0["Learning outcomes and connection"]
N1["6.1 Objections as hypotheses"]
N2["6.2 Boundary conditions"]
N3["6.3 The narrowed claim"]
N4["6.4 Dual-use review"]
N5["6.5 Responding to reviewers"]
N6["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX033"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["6.1 Objections as hypotheses"]
N2["6.2 Boundary conditions"]
N3["6.3 The narrowed claim"]
N4["6.4 Dual-use review"]
N5["6.5 Responding to reviewers"]
N6["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX033"]
[Learning outcomes and connection]
|
v
[6.1 Objections as hypotheses]
|
v
[6.2 Boundary conditions]
|
v
[6.3 The narrowed claim]
|
v
[6.4 Dual-use review]
|
v
[6.5 Responding to reviewers]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX033]
Learning outcomes and connection
After this chapter you can (1) organize an internal red-team review of your own claims, (2) convert objections into testable hypotheses, (3) characterize boundary conditions instead of hiding them, (4) perform a dual-use review of a paper before submission, and (5) write a responsive reviewer letter. It attacks the results of Chapter 5 and feeds the thesis argument of Chapter 7.
6.1 Objections as hypotheses
Invite two colleagues — ideally a control engineer and an offensive security researcher — to attack the claims. Record each objection as a hypothesis with a test:
| Objection (red team) | Hypothesis | Test | Outcome (synthetic) |
|---|---|---|---|
| "Your attacker is naive: a smart attacker ramps slowly." | A slow ramp below the CUSUM allowance evades detection while still overfilling. | Sweep attack gain from 0.005 to 0.05. | Gain 0.005 is detected in only 14/100 runs and 0.01 in 55/100, but worst overfill in the 300-step window stays at 1.12 and 1.37 units, against 3.36 at gain 0.05 — the impact-bounding argument of Urbina et al. |
| "The flow meter is just another PLC input." | If the meter is wired to the same PLC as the level sensor, class B is as easy as class A. | Inspect T2 wiring and Plant A design. | On T2 both pass through one PLC; the honest-sensor assumption holds only for a separately wired meter. |
| "Operators would notice the pump running." | Pump-state display reveals attack. | Review HMI design. | Only if pump state comes from an independent contact, not from the commanded value. |
| "Results depend on noise level." | Higher sensor noise raises false alarms at fixed k. | Lab 6 with noise doubled. | Requires retuning; reported as parametric sensitivity. |
The second objection is the most important: it reveals that the defense's strength is an architectural property (independent sensing paths), not an algorithmic one. The thesis claim is revised accordingly.
6.2 Boundary conditions
A good thesis maps where the method works. For the Setpoint Study the boundaries are: (1) an independent second measurement of the same physics must exist and be outside the attacker's control; (2) the model error must be smaller than the attack signal; (3) attacks slower than the allowance can evade, but their impact within a stated horizon is bounded (in the model, ≤1.37 units of overfill in 300 steps for gains ≤0.01). These are stated in the abstract, not buried in a limitations section.
Each boundary should be stated with the evidence that measures it: the T2 wiring inspection for the first, the calibration-bias sweep in Lab 6 for the second, and the attack-gain sweep from the red-team review for the third.
6.3 The narrowed claim
Before: "Physics-based CUSUM detects sensor manipulation in water plants." After: "Where an independently wired sensor measures a quantity linked by a known mass balance, a residual CUSUM with allowance tuned for expected calibration error detects single-sensor replay attacks that band alarms miss, and bounds the overfill achievable by slower attacks; it provides no protection when the attacker controls both sensing paths." The narrowed claim is weaker in reach and much stronger in evidence.
Keep the before and after versions in the thesis, so readers see how critique changed the argument. Check every later sentence against the narrowed claim: an abstract or conclusion that drifts back toward water plants in general undoes the work. The narrowed claim also gives operators a direct test of their own site: is the second measurement independently wired?
6.4 Dual-use review
Before submission ask: Does the paper give a recipe for the class B evasion against a named product? (No: it describes the capability class, which is already documented as manipulation of view.) Does it reveal T2's configuration? (Only generic structure.) Would a defender gain more than an attacker? (Yes: the architectural recommendation — independent sensing paths — is actionable for defenders and well known to capable attackers.) Record the answers in the thesis appendix.
Ask one reviewer outside the project to answer the same questions without seeing your answers, and record any disagreement and how it was resolved. If later work finds a flaw specific to a product, such as a PLC accepting writes it should refuse, the review is repeated and disclosure is handled under the coordinated process before publication.
6.5 Responding to reviewers
Answer every point, quote it, state what changed and where, and disagree with evidence where you must. Example: "Reviewer 2 states that CUSUM is not novel. We agree and have revised §1 to position the contribution as a measurement of when cross-sensor invariants help against manipulation of view, citing Page (1954) and Giraldo et al. (2018). The new contribution statement is on p. 3."
Keep a table with one row per reviewer point: the quotation, your response, the change made and its location. Where you disagree, cite the measured result, such as class B evading in every trial, rather than restating your opinion. Send the table with the revised manuscript so the editor can check every change.
6.6 Common errors and limits
Defending instead of testing; hiding the class B failure; claiming generality across plants after two contexts; responding to reviewers without stating changes.
Each error turns critique into defence. Answer the slow-ramp objection with the gain sweep, not with reassurance. Put class B in the abstract. Limit generality to the two contexts measured, the Plant A model and Testbed T2. In reviewer letters, state what changed and where, so a reader can verify the revision instead of trusting the claim. Keep the objections table in the thesis.
Exercises — chapter 6
- EX033 — Overfitting objection as a test. Turn the objection "your threshold is overfitted" into a hypothesis and a test. Rubric: 3 = hypothesis of drop on unseen seeds/T2 + tune 0–99 / eval 1000–1099 & T2; 2 = hypothesis without protocol; 1 = restate objection; 0 = none.
- EX034 — Sweep attack gain for CUSUM. Modify Lab 3 to sweep attack gain (0.005, 0.01, 0.02, 0.05) for CUSUM and report detection and worst overfill. Rubric: 3 = four gains with detect counts, delays, worst overfill matching lab sweep; 2 = two gains; 1 = qualitative; 0 = none.
- EX035 — Boundary paragraph for the abstract. Write the boundary-condition paragraph for the abstract in ≤ 80 words. Rubric: 3 = ≤80 words: independent second measurement, model error < attack, slow attacks ≤1.37 overfill; 2 = boundaries without numbers; 1 = overclaim; 0 = none.
- EX036 — Narrow the water-utilities claim. Rewrite the claim "the method secures water utilities" as a narrowed claim. Rubric: 3 = two synthetic contexts + independent flow + class B failure; 2 = narrowed but still 'utilities'; 1 = minor edit; 0 = original claim.
- EX037 — Dual-use on exact T2 spoof sequence. Conduct a dual-use review for publishing the exact flow-meter spoofing sequence used on T2. Rubric: 3 = publish class+defence; exact T2 sequence only under CVD/vendor; 2 = withhold without path; 1 = publish all; 0 = none.
- EX038 — Response to testbed representativeness. Write a reviewer response to "Testbed results are not representative of real plants.". Rubric: 3 = agree difference, point to narrowed claims + frozen thresholds on T2 + residual risk; 2 = agree without evidence pointers; 1 = dismiss; 0 = none.
Chapter synthesis
Critique is data: convert objections into tests, let them narrow the claim, publish boundary conditions prominently, and review dual-use before submission. The most valuable objection here turned an algorithmic claim into an architectural one.
Continue studying — chapter 06 of 08
← Chapter 05 (draft) · Course chapter index · Chapter 07 (draft) →