Chapter 38 — Quantum and post-quantum security: validation
Lesson map
Lesson 38 published. Map, example, EX075.
flowchart TD
N0["Learning outcomes"]
N1["38.1 Verification versus validation"]
N2["38.2 Acceptance tests"]
N3["38.3 Validating risk reduction"]
N4["38.4 Independent validation"]
N5["Engineering note"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX075"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["38.1 Verification versus validation"]
N2["38.2 Acceptance tests"]
N3["38.3 Validating risk reduction"]
N4["38.4 Independent validation"]
N5["Engineering note"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX075"]
[Learning outcomes]
|
v
[38.1 Verification versus validation]
|
v
[38.2 Acceptance tests]
|
v
[38.3 Validating risk reduction]
|
v
[38.4 Independent validation]
|
v
[Engineering note]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX075]
Learning outcomes
You will (1) validate that the migrated system meets stakeholder needs; (2) distinguish verification evidence from validation evidence; (3) run user-facing and partner acceptance tests; and (4) validate risk reduction claims carefully. It builds on Chapters 16, 30 and 37.
38.1 Verification versus validation
Verification shows requirements are met ("edge TLS negotiates X25519MLKEM768 with 99.2% of offering clients"). Validation shows the system serves its purpose ("shipping lines, customs and staff operate normally, and long-lived data flows are protected against recording"). Both are needed; one does not imply the other.
A system can pass verification and still fail validation: the edge may negotiate the hybrid group with 99.2% of offering clients while a partner's batch job quietly fails because its old client times out. Validation would catch that through the partner's own acceptance testing. Conversely, partners may be satisfied while a requirement is unmet, for example if replication still uses classical key exchange.
38.2 Acceptance tests
Partners run acceptance tests on their workflows; operations staff confirm no new incidents; legal confirms dual-signed documents are acceptable; device teams confirm firmware updates succeed across models. Record acceptance with dates and signatories.
Acceptance is recorded per stakeholder because each checks a different property: partners their workflows, legal the evidential value of signatures, device teams the update path. A dated signature from each makes later disputes simple to resolve. When a stakeholder cannot accept, the reason becomes a work item rather than being noted and forgotten.
38.3 Validating risk reduction
Claim carefully: "Recorded edge traffic negotiated with hybrid groups is protected against future quantum decryption if either X25519 or ML-KEM-768 remains secure" is defensible. "BPA is quantum-safe" is not — certificates, some VPNs, devices and unknown assets remain. Validation reports what is protected and what is not.
The defensible statement names exactly what is protected, under which assumption, and implicitly what is not. Listing the remaining classical paths with dates turns the gap into a plan instead of a hidden weakness. Boards can make decisions with such statements; a slogan like "quantum-safe" gives them nothing to act on and becomes embarrassing when an exception is discovered.
38.4 Independent validation
An independent team (internal audit or external assessor) samples flows, checks negotiated algorithms from their own vantage point and reviews the CBOM's unknowns. Independence catches optimistic self-assessment.
An independent team capturing traffic from its own vantage point may find, for example, that a service reported as hybrid actually negotiates classical groups for a particular client family. Sampling flows rather than relying on the team's dashboard is what makes the check independent. Reviewing unknowns matters because a self-assessment tends to treat them as minor. The assessor's findings go into the dossier with the team's responses.
38.5 Worked example
BPA's validation report: 64% of edge connections hybrid (verification); partner acceptance complete for 11 of 14 lines (validation); customs exports dual-recipient (validation pending agency test); firmware dual-key bootloader on 40% of devices; unknown assets 6%. Statement of residual risk lists the remaining classical paths with dates.
The report separates verification from validation on each line, so readers can see which claims rest on telemetry and which on stakeholder confirmation. The pending customs test is reported as pending, not as complete, and the 40% firmware figure states device coverage, not image signing. That precision is what lets the board trust the rest of the report.
38.6 Common errors and limits
- Treating verification metrics as validation.
- Overclaiming "quantum-safe".
- Self-validation only.
A high hybrid share is a verification result; it says nothing about whether partners can still work, which only validation confirms. Overclaiming "quantum-safe" ignores the classical certificates, VPNs and unknown assets the report lists. Self-validation alone tends to miss what the team did not think to check, which is why an independent sample is part of BPA's process. Each of these errors appears in early migration reports from many organizations, usually discovered by an auditor rather than the team.
Engineering note
Validation should revisit the original stakeholder needs from Chapter 21 one by one. For each, state whether it is met, partly met or not met, with evidence, and record any need that changed during the programme — for example, a partner whose data retention requirement grew from five to ten years after a contract renewal. Needs drift; validation that checks against the original list alone can declare success on a target that no longer reflects what stakeholders actually require.
Exercises
- EX075 — Residual risk. Write the residual-risk statement for BPA after year one. Rubric: 3 = lists unprotected paths with dates and owners.
- EX076 — Independence. Design an independent validation sample for edge TLS. Rubric: 3 = vantage point, sample size, comparison with internal metrics.
Synthesis
Validation confirms the migrated system serves its stakeholders and states residual risk honestly, ideally checked by an independent team.
Continue studying — chapter 38 of 50
← Chapter 37 (draft) · Course chapter index · Chapter 39 (draft) →