Chapter 37 — Evidence-review and case-critique seminar | Bachelor's Incident response and cyber resilience

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 37 — Evidence-review and case-critique seminar

Lesson map

Lesson 37 published. Map, example, EX109.

flowchart TD
  N0["Learning outcomes and connection"]
  N1["37.1 The purpose of critique"]
  N2["37.2 A critique protocol"]
  N3["37.3 Leading the seminar"]
  N4["37.4 Revision"]
  N5["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5
  N5 --> EX["Worked example"]
  EX --> Q["Exercise EX109"]

Logical drawing

flowchart LR
  N0["Learning outcomes and connection"]
  N1["37.1 The purpose of critique"]
  N2["37.2 A critique protocol"]
  N3["37.3 Leading the seminar"]
  N4["37.4 Revision"]
  N5["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX109"]

[Learning outcomes and connection]
    |
    v
[37.1 The purpose of critique]
    |
    v
[37.2 A critique protocol]
    |
    v
[37.3 Leading the seminar]
    |
    v
[37.4 Revision]
    |
    v
[Chapter synthesis]
    |
    v
[Worked example] --> [Exercise EX109]

Learning outcomes and connection

You will be able to (1) assess the sources in a case dossier; (2) generate alternative explanations a dossier ignored; (3) give and receive structured peer critique; and (4) revise a conclusion with evidence. This seminar applies source grading (Chapter 20), provenance (Chapter 22) and hypothesis testing (Chapter 23) to dossiers written by students, and prepares the capstone evaluation (Chapter 39).

37.1 The purpose of critique

Critique tests whether a conclusion survives scrutiny. It is directed at claims and evidence, not at people. A dossier that survives critique is stronger; one that changes is better. Professional peer review in incident response (for example, a second analyst checking a timeline before a report goes to management) serves the same purpose.

In the dossier example, Dossier 1 changed substantially under critique and became better for it; its revised conclusion keeps the account-takeover finding but drops the claims about a criminal group, exfiltration and fraud that the bundle could not support. That is what success looks like in a seminar.

37.2 A critique protocol

For each conclusion: (1) Which evidence supports it, and what grade is that evidence? (2) Which transformations sit between source and claim, and are they reproducible? (3) What alternative explanations exist, and were they tested? (4) Is the wording calibrated to the evidence? (5) What single piece of additional evidence would most change the conclusion?

Applied to Dossier 1, the second question exposes the word "exfiltrated": the bundle contains an application export of 1,820 rows but no network evidence of transfer. The fifth question points to the evidence most worth seeking, such as whether the export file was downloaded or emailed. Writing answers to all five questions for each conclusion keeps critics from fixing on style.

37.3 Leading the seminar

The leader assigns roles: presenter, two critics, a scribe. Presenter states conclusions (5 minutes); critics apply the protocol (10 minutes); presenter responds (5 minutes); scribe records agreed revisions. Rules: cite page or line; no ad hominem; disagreements are recorded rather than forced to consensus.

Keep time strictly; critics who run over squeeze the presenter's response, which is where revision begins. The scribe should record the agreed revisions in the presenter's own words and note any disagreement that remained, with the evidence each side cited. Rotate roles across sessions so each student presents, critiques and records at least once. The leader should intervene only to enforce the rules, not to settle the substance.

37.4 Revision

The presenter revises: changes wording, adds tests, or withdraws conclusions. A revision log shows what changed and why.

For Dossier 1, the revision log would show "criminal group" removed for lack of attribution evidence, "exfiltrated" replaced by the recorded export with a note that no transfer was observed, and the fraud motive withdrawn. Each entry cites the critique point and the evidence behind the change. Keep the original text in the log; a reader should be able to see what was claimed before and why it no longer is. A dossier that changed nothing after critique should explain why each challenge did not require a change.

37.5 Worked example (synthetic): two dossiers, same events

Fictional. Two students analysed the same synthetic bundle (the Chapter 25 events plus a help-desk ticket).

Dossier 1 concludes: "The attacker, likely a criminal group, exfiltrated 1,820 supplier records and stole the supplier-7 password to commit invoice fraud." Dossier 2 concludes: "Likely account takeover of fin-04; an export of 1,820 rows and a view of the supplier-7 stored credential occurred during the session; no evidence in the bundle shows data leaving the network or any fraud."

Critique of Dossier 1: "criminal group" — no evidence of attribution (unsupported). "Exfiltrated" — the export is an application action; no network evidence of transfer (overclaim). "To commit invoice fraud" — motive inferred, not observed. Alternative explanation not considered: fin-04 legitimately exported records for month-end and viewed the credential, with the account takeover occurring separately — testable by asking the user and checking whether the export file was downloaded or emailed.

Dossier 2 is better calibrated, but critics note that "likely account takeover" relies on the user's denial and an MFA change; it should state the competing explanation explicitly and the evidence that would discriminate (device used, file download).

Revised Dossier 1 conclusion: "Likely account takeover of fin-04 (MFA method added 07:41 after sign-in from an unrecognized session; user denies activity). During the session, an export of 1,820 supplier rows and a view of a stored supplier credential were recorded. Whether the export left the organization and whether these actions were the attacker's or the user's is not established."

37.6 Common errors and limits

Critiquing style instead of claims.
Accepting the presenter's framing of alternatives.
Forcing consensus where evidence is genuinely ambiguous.
Peer critique among students lacks the independence and expertise of professional review; note this.
Critics proposing alternative explanations without saying what evidence would test them.
Presenters defending every sentence instead of revising where the critique is sound.
Treating the revised dossier as final when the most discriminating evidence, such as whether the export left the organization, has still not been sought.
Letting the most confident speaker settle the seminar.

Exercises — chapter 37

Attempt each exercise before opening the worked results. Every case, organization, person, address and number in an exercise is synthetic unless a cited public source is named.

EX109 — Apply the protocol. Steps: apply the five questions to Dossier 2. Rubric: 3 = all five answered with specific references; 2 = four; 1 = generic; 0 = none.

EX110 — Discriminating evidence. Steps: name the single piece of evidence that would most change Dossier 2's conclusion and explain why. Rubric: 3 = e.g., evidence of the session's device and source (user's managed laptop vs unknown host) or proxy/mail logs showing the export file leaving; explains which hypothesis each outcome supports; 2 = evidence without reasoning; 1 = vague; 0 = none.

EX111 — Revision log. Steps: write a revision log for Dossier 1 with at least three entries. Rubric: 3 = original text, revised text, reason, evidence reference; 2 = missing reasons; 1 = summary only; 0 = none.

Chapter synthesis

Critique claims with a fixed protocol, insist on alternatives and discriminating evidence, record disagreements honestly and keep a revision log so improvement is visible.

Continue studying — chapter 37 of 40

← Chapter 36 · Course chapter index · Chapter 38 →

Continue to the next authored lesson.