Chapter 6 — Peer critique and revision
Lesson map
Lesson 06 published. Map, example, EX033.
flowchart TD
N0["Learning outcomes"]
N1["6.1 The critique"]
N2["6.2 Responding to name variation"]
N3["6.3 Responding to interview bias"]
N4["6.4 Limiting the documentation claim"]
N0 --> N1 --> N2 --> N3 --> N4
N4 --> EX["Worked example"]
EX --> Q["Exercise EX033"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["6.1 The critique"]
N2["6.2 Responding to name variation"]
N3["6.3 Responding to interview bias"]
N4["6.4 Limiting the documentation claim"]
N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX033"]
[Learning outcomes]
|
v
[6.1 The critique]
|
v
[6.2 Responding to name variation]
|
v
[6.3 Responding to interview bias]
|
v
[6.4 Limiting the documentation claim]
|
v
[Worked example] --> [Exercise EX033]
Learning outcomes
You will (1) respond to peer critique of governance research; (2) revise analyses transparently; (3) strengthen limitations.
6.1 The critique
Nadia presents her results at a doctoral workshop. Peers raise three critiques. First, the synthetic replica may understate real-world name variation, which would change linkage error. Second, interviews with Westmark staff may reflect blame avoidance, since nobody wants to admit owning a poorly chosen threshold. Third, the claim that documentation reduces variation rests on only two releases. Prof. Eriksen tells her that good critique is a gift, because it identifies weaknesses before examiners or users find them. She writes each critique down verbatim.
6.2 Responding to name variation
Nadia adds a sensitivity analysis in which name agreement is less reliable, lowering m for names from 0.95 to 0.85. Linkage error rises at every threshold, but the ranking of decisions by effect remains the same. She reports both versions and explains that the size of effects is uncertain while the ranking is robust. She also notes that NIST SP 800-188 discusses how synthetic data may preserve some properties and not others, so analysts must state which properties they relied on. Both analyses are archived together.
6.3 Responding to interview bias
To address possible blame avoidance, Nadia compares interview answers with documents such as release notes, meeting minutes and change requests. Documents confirm that no record exists of a threshold review since the original choice. They also show that the methods team proposed a review two years ago that was never scheduled. Triangulation strengthens the finding that ownership was unclear, while softening any suggestion that individuals ignored the issue. She revises her wording to describe a structural gap rather than personal failure. Documents settle it.
6.4 Limiting the documentation claim
With only two releases, Nadia cannot show that documentation causes reduced variation. She rewrites the third claim as an observation consistent with the hypothesis and proposes a follow-up design in which Westmark documents decisions for four releases while a comparison series continues without documentation. Prof. Eriksen approves the revision. Nadia records every change in her decision log, including the original wording, so that examiners can see how the thesis responded to critique rather than quietly shifting its claims. Examiners value this honesty highly.
Worked example
Nadia answers critiques with a sensitivity analysis on name agreement, triangulates interviews with documents and reframes the documentation claim as consistent but not causal.
Synthetic case. The workshop is synthetic; NIST SP 800-188 is real.
Errors and limits
Sensitivity analyses test chosen alternatives only; others may matter.
Exercises
- EX033 — Name critique. What critique concerned names? Rubric: 3 = replica may understate name variation;; 2 = partly correct;; 1 = vague;; 0 = none.
- EX034 — Sensitivity change. How did Nadia test name variation? Rubric: 3 = lowered m for names from 0.95 to 0.85;; 2 = partly correct;; 1 = vague;; 0 = none.
- EX035 — Robust result. What stayed the same in the sensitivity analysis? Rubric: 3 = the ranking of decisions by effect;; 2 = partly correct;; 1 = sizes;; 0 = none.
- EX036 — Triangulation. What did documents show about threshold reviews? Rubric: 3 = none since the original choice; a proposal never scheduled;; 2 = both;; 1 = one;; 0 = no answer.
- EX037 — Structural wording. How did Nadia revise her wording on ownership? Rubric: 3 = a structural gap, not personal failure;; 2 = partly correct;; 1 = blame;; 0 = none.
- EX038 — Follow-up design. What follow-up did Nadia propose? Rubric: 3 = four documented releases against a comparison series;; 2 = mentions comparison;; 1 = vague;; 0 = none.
Continue studying — lesson 06 (chapter 06 of 08)
← Chapter 05 · Course chapter index · Chapter 07 →
Continue to the next authored lesson.