Chapter 48 — Evidence dossier

Chapter 48 — Evidence dossier

Lesson map

Lesson 48 published. Map, example, EX095.

flowchart TD
  N0["Learning outcomes and connection"]
  N1["48.1 What the dossier is"]
  N2["48.2 Provenance"]
  N3["48.3 Sensitivity"]
  N4["48.4 Consistency check"]
  N5["48.6 Practical activity"]
  N6["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX095"]

Logical drawing

flowchart LR
  N0["Learning outcomes and connection"]
  N1["48.1 What the dossier is"]
  N2["48.2 Provenance"]
  N3["48.3 Sensitivity"]
  N4["48.4 Consistency check"]
  N5["48.6 Practical activity"]
  N6["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX095"]

[Learning outcomes and connection]
    |
    v
[48.1 What the dossier is]
    |
    v
[48.2 Provenance]
    |
    v
[48.3 Sensitivity]
    |
    v
[48.4 Consistency check]
    |
    v
[48.6 Practical activity]
    |
    v
[Chapter synthesis]
    |
    v
[Worked example] --> [Exercise EX095]

Learning outcomes and connection

After this chapter you can (1) assemble an engineering evidence dossier that supports the capstone's claims; (2) organise evidence by claim with provenance; (3) protect sensitive evidence while keeping it reviewable; and (4) check the dossier for consistency before submission. It gathers Chapters 44–47 and prepares the engineering thesis.

48.1 What the dossier is

The evidence dossier is the collection of artefacts that supports the assurance case: requirements and traceability; design documents and decision records; analyses (threat models, fault trees, performance, reliability); implementation records (source revision, SBOM, build provenance); verification and validation results; review findings and responses; trial reports; operational documents; and the residual-risk register with owners. Organise it by claim, not by date: a reviewer should be able to start from a claim and find its evidence in two clicks.

For TMRP, the dossier's claims are C1 to C4.

48.2 Provenance

Every piece of evidence states what it is, who produced it, when, from which version of the system and with which tools. Numbers in the thesis must trace to a result file (compare the master's provenance check in its Chapter 7). Use content hashes for result files and record them in an index, as Chapter 13's hash chain does for logs.

For TMRP, each entry in the dossier index carries a version, such as the gateway release or the trial date, and the first characters of its hash.

48.3 Sensitivity

Dossiers about infrastructure contain sensitive details. Keep a full restricted version for authorised reviewers and a redacted version for wider audiences, and record what was redacted and why. For coursework with synthetic cases, sensitivity is lower — but practise the discipline.

For TMRP, the restricted version would include gateway configurations and key-management procedures, while the redacted version describes the architecture at the level of zones and conduits. The redaction record lists each removed item and the reason, so a reviewer of the redacted version knows what was withheld and can ask for access through the proper route.

48.4 Consistency check

Before submission: every claim has evidence; every number in the thesis matches its source; statuses agree across documents (no "passed" in one and "failed" in another — finding F1 in Chapter 45); residual risks have owners; and all review findings are closed or dispositioned.

TMRP's dossier failed this check once, when C2 cited T3 as passed while the matrix still showed it failed. The fix was to record the v2 re-test as T3b and update every document that referred to it, so the index now shows the same status everywhere.

48.5 Worked example: dossier index excerpt (synthetic)

(The hashes are illustrative placeholders for the synthetic case.)

48.6 Practical activity

Build your dossier index with real content hashes of your result files (hashlib.sha256) and run the §48.4 consistency check.

Use the script from EX095 to compute the hashes, and store the index under version control so its history is part of the evidence. Then run the consistency check against your thesis draft. Every number in the draft should appear in a result file listed in the index, and every status should match. Fix any mismatch in the source document first, then regenerate the index.

48.7 Common errors and limits

• Dossiers organised by date or author instead of claim.
• Numbers in the thesis with no source file.
• Status inconsistencies across documents.
• Redaction without a record.

The TMRP index avoids the first error by organising evidence by claim, and its record of C3 as not met keeps the third from appearing. A redaction record accompanies the shared version. Every number in the thesis traces to a result file listed in the index, and the shared version carries a record of what was redacted and why.

Exercises — chapter 48

EX095 — Index script. Objective: automate provenance. Inputs: a results folder. Steps: write a stdlib script that walks the folder, computes SHA-256 of each file, and writes a CSV index with path, size, hash and modification time; explain why modification time is weak evidence. Expected evidence: script and explanation. Rubric: 3 = working script, explanation (mtime is easily changed; hash plus version control commit is stronger); 2 = no explanation; 1 = not working; 0 = none.

EX096 — Consistency audit. Objective: catch inconsistencies. Inputs: the §48.5 excerpt and an abstract sentence "All requirements were verified and the system restores remote operation within 8 hours". Steps: identify the inconsistency and rewrite the sentence. Expected evidence: rewrite. Rubric: 3 = identifies C3 contradiction, rewrite accurate and still positive where justified; 2 = vague rewrite; 1 = misses it; 0 = none.

Chapter synthesis

The dossier is the claim-organised, provenance-tracked body of evidence behind the assurance case. Keep restricted and redacted versions with a redaction record, and check consistency of every claim, number and status before anyone else does.

Continue studying — chapter 48 of 50

← Chapter 47 · Course chapter index · Chapter 49 →

Continue to the next authored lesson.