Chapter 48 — Evidence dossier

Lesson map

Lesson 48 published. Map, example, EX142.

flowchart TD
  N0["Learning outcomes"]
  N1["48.1 Assembling the dossier"]
  N2["48.2 Claims and evidence"]
  N3["Lab — Claims by evidence strength"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX142"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["48.1 Assembling the dossier"]
  N2["48.2 Claims and evidence"]
  N3["Lab — Claims by evidence strength"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX142"]

[Learning outcomes]
    |
    v
[48.1 Assembling the dossier]
    |
    v
[48.2 Claims and evidence]
    |
    v
[Lab — Claims by evidence strength]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX142]

Learning outcomes

You will (1) assemble an evidence dossier for the programme; (2) link claims to evidence; (3) state limitations and open questions.

48.1 Assembling the dossier

Aiko assembles an evidence dossier for her engineering assessment and for Greyford's regulator. It contains the requirements, design rationale, verification tables, validation results, pilot analysis, failure trial report, review log and ethical rules. Each document is versioned and dated. The dossier is organised around claims, such as the claim that the programme shortens exposure to malicious emails, with the evidence for each claim listed beneath it. This structure helps readers see what is claimed and how strongly it is supported. An index lists every item.

48.2 Claims and evidence

For each claim, Aiko states the evidence and its strength. The exposure claim rests on the before and after comparison and the queue analysis, which she rates as moderate because the design cannot rule out changes in attacks. The payment fraud claim rests on caught attempts and the reliability analysis. The fairness claim for contractors rests on the pilot and interviews. The lab counts claims by evidence strength. She avoids claiming that the programme changed culture, because her evidence does not measure culture directly.

Lab — Claims by evidence strength

# Claims by evidence strength (synthetic).
claims = {"shorter exposure": "moderate", "payment fraud caught": "strong", "contractor fairness": "moderate", "low staff burden": "moderate", "lasting effect": "weak"}
for s in ("strong", "moderate", "weak"): print(s, sum(v == s for v in claims.values()))

Run with python3 lab_dossier.py. Verified output:

strong 1
moderate 3
weak 1

48.3 Limits and open questions

The dossier ends with limitations and open questions: whether the effects last beyond two years, whether results would hold in a larger utility and whether feedback wording matters, given the inconclusive study. She lists data that would answer each question. The panel praises the dossier for its honesty. Prof. Kerr reminds her that a dossier is a living record and should be updated as the programme continues, so that later decisions rest on current evidence rather than the snapshot taken for the assessment.

Worked example

Aiko builds a claim-based dossier, rates evidence strength and lists open questions.

Synthetic case. The dossier is synthetic.

Errors and limits

Strength ratings are judgements. Dossiers become outdated.

Exercises

• EX142 — Organisation. How is the dossier organised?
  Show marking criteria

  Rubric: 3 = around claims with evidence beneath each;; 2 = claims;; 1 = by date;; 0 = none.
• EX143 — Lab strengths. What did the lab count?
  Show marking criteria

  Rubric: 3 = 1 strong, 3 moderate, 1 weak claim;; 2 = counts;; 1 = vague;; 0 = none.
• EX144 — Avoided claim. Which claim did Aiko avoid?
  Show marking criteria

  Rubric: 3 = that the programme changed culture;; 2 = culture;; 1 = vague;; 0 = none.

Synthesis

An evidence dossier links each claim to graded evidence and names its limits.

Continue studying — chapter 48 of 50

← Chapter 47 (draft) · Course chapter index · Chapter 49 (draft) →