Chapter 48 — Evidence dossier
Lesson map
Lesson 48 published. Map, example, EX095.
flowchart TD
N0["Learning outcomes"]
N1["48.1 Contents"]
N2["48.2 Linking claims to evidence"]
N3["48.3 Protecting evidence"]
N4["48.4 Reviewability"]
N5["Engineering note"]
N6["48.7 Keeping the dossier current"]
N7["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX095"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["48.1 Contents"]
N2["48.2 Linking claims to evidence"]
N3["48.3 Protecting evidence"]
N4["48.4 Reviewability"]
N5["Engineering note"]
N6["48.7 Keeping the dossier current"]
N7["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX095"]
[Learning outcomes]
|
v
[48.1 Contents]
|
v
[48.2 Linking claims to evidence]
|
v
[48.3 Protecting evidence]
|
v
[48.4 Reviewability]
|
v
[Engineering note]
|
v
[48.7 Keeping the dossier current]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX095]
Learning outcomes
You will (1) assemble an evidence dossier for the migrated system; (2) link evidence to requirements and claims; (3) protect sensitive evidence; and (4) make the dossier reviewable by auditors and successors. It prepares the thesis and defense.
48.1 Contents
Requirements and traceability matrix; architecture and decision records; CBOM snapshots with method; threat and FMEA analyses; test reports (KATs, interoperability, downgrade, state); measurement data (negotiated groups, failures, latency); validation and acceptance records; failure-trial reports; residual-risk statement; agility demonstration results.
Each item type answers a different reviewer question: the matrix shows what was required, test reports show what was checked, measurement data shows what happened in operation, and the residual-risk statement shows what remains. A reviewer should be able to start from any claim in the summary and reach its evidence through the index.
48.2 Linking claims to evidence
Every claim in summaries has an evidence identifier. "Edge connections negotiating hybrid groups: 71% (EV-MEAS-14, 30-day window, passive telemetry)". Unlinked claims are removed.
The identifier tells the reader where the evidence is, the window tells them when it was measured, and the source tells them how. Removing unlinked claims may make a summary shorter, but every remaining sentence can be checked. The auditor's sample of ten claims in the dossier example shows how this works in practice, with each claim traced in minutes.
48.3 Protecting evidence
Evidence can reveal weaknesses. Classify dossier parts; share sensitive parts (vulnerability details, network maps) only with authorized reviewers; publish only sanitized summaries.
A network map showing which flows still use classical key exchange is useful to an attacker planning to record traffic, so it is classified. Sanitized summaries replace system names with categories and give totals rather than lists. Access to classified parts is logged, and the dossier index shows which parts exist without exposing their contents. BPA reviews classifications when the dossier is updated, since a gap that is closed may no longer need protection.
48.4 Reviewability
Use a consistent structure, index and version control. A successor should find in minutes where ML-DSA keys live and how to replace them.
BPA's dossier follows the same order as this book's verification chapters, with an index linking each evidence identifier to its file. Version control shows when each item changed and who changed it. A new engineer's first task is to answer three questions from the dossier, such as where the ML-DSA keys live, which tests whether the structure works.
48.5 Worked example
BPA's dossier index lists 126 evidence items. An auditor samples ten claims and traces each to evidence in under an hour; one claim — "all firmware dual-signed" — is found to apply only to devices with the new bootloader; the claim is corrected to "all images dual-signed; 40% of devices verify both".
The corrected claim is more useful than the original: it tells operations that 60% of devices still verify only one signature, which is a real residual risk. Sampling ten claims in under an hour also shows the dossier is reviewable, the property the previous part asked for.
48.6 Common errors and limits
- Claims without evidence identifiers.
- Dossiers with sensitive details shared broadly.
- Evidence scattered across personal folders.
Claims without identifiers cannot be checked and invite overstatement, as the firmware claim showed. Sharing sensitive parts broadly hands attackers a map of remaining weaknesses. Evidence in personal folders disappears when people leave, so the next migration starts without it. BPA keeps all evidence in the versioned repository with access controls, and the index is checked for broken links and missing items before each audit.
Engineering note
Dossiers also need a maintenance owner. Evidence ages: measurements become stale, standards statuses change, partners upgrade. Assign an owner, a review frequency and an expiry date to each evidence item, so that the dossier remains a truthful description of the system rather than a snapshot of the week it was submitted. Auditors trust dossiers that show their own update history.
48.7 Keeping the dossier current
BPA's dossier is versioned: each update states what changed, why and who approved it. When a standard is revised or a vendor ships a new post-quantum option, the affected claims are re-checked and their evidence refreshed, and stale items are marked rather than silently removed.
A stale item stays visible with its date and the reason it lapsed, so a reviewer can tell an old success from a current one. When FIPS 206 for FN-DSA is final, for example, the claim that FN-DSA is excluded pending standardization is reviewed, and the decision log records whether BPA now adopts it for constrained links.
Exercises
- EX095 — Index. Draft the dossier index structure. Rubric: 3 = categories, identifiers, classification.
- EX096 — Claim audit. Audit five claims from Chapter 46's results against evidence needs. Rubric: 3 = each linked or corrected.
Synthesis
An evidence dossier links every claim to protected, indexed evidence that auditors and successors can navigate.
Continue studying — chapter 48 of 50
← Chapter 47 (draft) · Course chapter index · Chapter 49 (draft) →