Chapter 6 — Peer critique and revision

Chapter 6 — Peer critique and revision

Lesson map

Lesson 06 published. Map, example, EX033.

flowchart TD
  N0["Learning outcomes"]
  N1["6.1 Seeking critique"]
  N2["6.2 Challenges raised"]
  N3["6.3 Responses"]
  N4["6.4 Revising claims"]
  N5["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5
  N5 --> EX["Worked example"]
  EX --> Q["Exercise EX033"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["6.1 Seeking critique"]
  N2["6.2 Challenges raised"]
  N3["6.3 Responses"]
  N4["6.4 Revising claims"]
  N5["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX033"]

[Learning outcomes]
    |
    v
[6.1 Seeking critique]
    |
    v
[6.2 Challenges raised]
    |
    v
[6.3 Responses]
    |
    v
[6.4 Revising claims]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX033]

Learning outcomes

You will (1) invite structured critique; (2) respond to methodological challenges; (3) revise claims and analyses.

6.1 Seeking critique

Lucía presents her results to a seminar of researchers and two external practitioners, and invites written critique against a short template: threats to validity, alternative explanations and missing analyses. Seeking critique early, before the thesis is written, gives time to respond with analysis rather than argument. She records every point with its source and her planned response, so that the thesis can show examiners how criticism shaped the work. She also asks one critic to argue deliberately against her conclusion, so that the strongest objections are heard before examiners raise them.

6.2 Challenges raised

Critics raise four points. First, seeded weaknesses come from public registries, which checklist items also reference, so the checklist may simply list what was planted. Second, auditors may have tried harder under the novel procedure. Third, the blinded reviewer might recognise checklist reports by their format. Fourth, twelve contracts per arm is small. Each is a fair challenge, and Lucía notes that the first is the most serious because it questions whether the comparison is fair at all. She thanks each critic and records the challenge verbatim.

6.3 Responses

For the first point, Lucía analyses the subset of residual issues that are business-logic or oracle problems not named on the checklist; the checklist arm still has fewer, though the difference is smaller. For the second, she reports that review hours were recorded and adds effort as a covariate. For the third, she shows reports were reformatted to a common template before blinded review. For the fourth, she agrees, reports intervals and calls for replication. Responding with analysis strengthens claims more than rebuttal.

6.4 Revising claims

After critique, Lucía revises her central claim: structured review was associated with fewer residual issues, with a smaller but consistent difference for issues not named on the checklist, suggesting that structure improves attention generally as well as coverage of listed items. She moves the strong version of the claim into limitations. Revising openly shows intellectual honesty and gives examiners confidence that the thesis reflects the evidence rather than her hopes. Her supervisor checks that every revised sentence matches the analyses, and the critique log is attached to the thesis as an appendix so examiners can trace each change.

Worked example

Four critiques are logged; analyses of unlisted issues, effort covariates and formatting controls are added, and the central claim is narrowed accordingly.

Synthetic case. Seminar participants, critiques and analyses are synthetic.

Errors and limits

Answering critique with argument alone, without new analysis, leaves doubts unresolved.

Exercises

• EX033 — Critique template. What does the critique template ask for? Rubric: 3 = threats, alternative explanations, missing analyses;; 2 = validity;; 1 = praise;; 0 = none.
• EX034 — Most serious. Which critique is most serious? Rubric: 3 = checklist may list what was planted;; 2 = seeding overlap;; 1 = sample size;; 0 = none.
• EX035 — Unlisted analysis. What did the unlisted-issue analysis show? Rubric: 3 = checklist still fewer, smaller difference;; 2 = smaller;; 1 = reversed;; 0 = none.
• EX036 — Format control. How was format recognition addressed? Rubric: 3 = reports reformatted to a common template;; 2 = common template;; 1 = ignored;; 0 = none.
• EX037 — Revised claim. How was the claim revised? Rubric: 3 = association with smaller consistent difference for unlisted issues;; 2 = narrowed;; 1 = strengthened;; 0 = none.
• EX038 — Small sample reply. How did Lucía respond to small samples? Rubric: 3 = report intervals and call for replication;; 2 = replication;; 1 = ignore;; 0 = none.

Synthesis

Structured critique, analytic responses and openly revised claims make a thesis stronger and more credible.

Continue studying — chapter 06 of 08

← Chapter 05 (draft) · Course chapter index · Chapter 07 (draft) →