Chapter 6 — Peer critique and revision
Lesson map
Lesson 06 published. Map, example, EX033.
flowchart TD
N0["Learning outcomes"]
N1["6.1 Inviting hostile reading"]
N2["6.2 Classifying comments"]
N3["6.3 Testing the fatal objection"]
N4["6.4 Revising claims"]
N5["6.5 Documenting revisions"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX033"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["6.1 Inviting hostile reading"]
N2["6.2 Classifying comments"]
N3["6.3 Testing the fatal objection"]
N4["6.4 Revising claims"]
N5["6.5 Documenting revisions"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX033"]
[Learning outcomes]
|
v
[6.1 Inviting hostile reading]
|
v
[6.2 Classifying comments]
|
v
[6.3 Testing the fatal objection]
|
v
[6.4 Revising claims]
|
v
[6.5 Documenting revisions]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX033]
Learning outcomes
You will (1) seek critique that could overturn your conclusions; (2) classify reviewer comments by their consequence; (3) revise claims, not just wording; (4) document revisions transparently.
6.1 Inviting hostile reading
Amara circulates her results chapter to three readers chosen to disagree: an empirical software engineering researcher sceptical of observational designs, a practising enterprise architect who uses SABSA and doubts that defect counts capture architecture quality, and a threat-modelling advocate who suspects the effect is underestimated. She asks each to identify the weakest claim and the analysis that could overturn it. Inviting critique from people with different commitments is a practical defence against the comfortable blindness that comes from discussing work only with supportive colleagues and a supervisor who has watched it grow.
6.2 Classifying comments
Amara sorts comments into three groups. Fatal comments, if correct, would invalidate the main claim. Material comments would change an estimate or an interpretation. Presentational comments affect clarity. The sceptical researcher's point that adoption timing might follow team performance is potentially fatal, because teams adopting after a bad year would show regression to the mean. The architect's point that defect counts ignore design quality is material for interpretation. Several comments about figure labels are presentational. Sorting this way ensures she spends effort where it matters, rather than polishing prose while a fatal objection waits.
6.3 Testing the fatal objection
To test the regression-to-the-mean objection, Amara compares each adopting team's defect rate in the year before adoption with its longer history. If teams adopted after unusually bad years, their pre-adoption rates would sit above their own averages. She finds adopting teams' pre-adoption rates within normal variation, and comparison teams show similar patterns. She also reruns the estimate using a two-year pre-period instead of one, which changes it only slightly. The objection is not refuted absolutely, but its predicted signature is absent. She reports the test and its result, crediting the reviewer by name with permission.
6.4 Revising claims
The architect's comment leads to a substantive revision. Amara had written that threat modelling improves architecture security. She now writes that it was associated with fewer post-release architecture defects as coded in the study, and adds a paragraph on what defect counts miss: designs that are secure but brittle, or defects in systems never released. She also adds a qualitative appendix with interviews from six architects about how sessions changed their designs. The claim is now smaller but defensible, and the appendix gives readers another kind of evidence to weigh against the counts.
6.5 Documenting revisions
Every change after critique goes into a revision table with the comment, its classification, the response and the location of the change. Where Amara disagrees, she explains why, for example declining the advocate's request to drop low-fidelity teams from the main analysis because that would bias the estimate upward. The table becomes an appendix. Prof. Quintero notes that examiners read revision tables closely because they show how a candidate handles disagreement. Transparent revision distinguishes scholarly confidence, which rests on tested objections, from mere assertiveness, which rests on not having asked.
Worked example
Amara collects critique from three opposed readers, classifies comments as fatal, material or presentational, tests regression to the mean, narrows her central claim and publishes a revision table.
Synthetic case. Reviewers, comments and tests are synthetic.
Errors and limits
Absence of an objection's signature is not proof it is wrong. Reviewers chosen by the author may still share blind spots.
Exercises
- EX033 — Reader selection. Why choose readers who disagree with each other? Rubric: 3 = different commitments expose different weaknesses and counter comfortable blindness;; 2 = diversity;; 1 = supportive readers;; 0 = none.
- EX034 — Fatal objection. Which objection was potentially fatal and why? Rubric: 3 = adoption after bad years causes regression to the mean that mimics an effect;; 2 = named;; 1 = presentational;; 0 = none.
- EX035 — Regression signature. What signature would regression to the mean leave, and was it found? Rubric: 3 = pre-adoption rates above teams' own averages; not found;; 2 = signature only;; 1 = found;; 0 = none.
- EX036 — Narrowed claim. How did Amara rewrite her central claim? Rubric: 3 = associated with fewer post-release architecture defects as coded, not improves security;; 2 = narrower;; 1 = unchanged;; 0 = none.
- EX037 — Declined request. Which reviewer request did she decline and why? Rubric: 3 = dropping low-fidelity teams, as it would bias the estimate upward;; 2 = declined;; 1 = accepted;; 0 = none.
- EX038 — Revision table value. Why do examiners read revision tables closely? Rubric: 3 = they show how the candidate handles disagreement;; 2 = disagreement;; 1 = formatting;; 0 = none.
Synthesis
Critique is most valuable when invited from opponents, sorted by consequence and answered by tests and narrower claims.
Continue studying — chapter 06 of 08
← Chapter 05 (draft) · Course chapter index · Chapter 07 (draft) →