Chapter 40 — Verification laboratory II
Lesson map
Lesson 40 published. Map, example, EX118.
flowchart TD
N0["Learning outcomes"]
N1["40.1 Broader scope"]
N2["40.2 Failure path tests"]
N3["40.3 Regression tests"]
N4["40.4 Final evidence"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX118"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["40.1 Broader scope"]
N2["40.2 Failure path tests"]
N3["40.3 Regression tests"]
N4["40.4 Final evidence"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX118"]
[Learning outcomes]
|
v
[40.1 Broader scope]
|
v
[40.2 Failure path tests]
|
v
[40.3 Regression tests]
|
v
[40.4 Final evidence]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX118]
Learning outcomes
You will (1) run a second, broader verification round; (2) test failure and recovery paths; (3) produce final verification evidence.
40.1 Broader scope
The second verification round extends beyond the first round's functional checks. It adds failure and recovery tests, such as HR feed interruption and connector failure, security regression tests for every earlier failure, tests of emergency access and tests of the improved access review. Sana updates the test catalogue from the traceability matrix so every must and should requirement has at least one test. The round runs in the test tenant using the current configuration version from the repository. Results are compared with the first round.
40.2 Failure path tests
Failure tests check that the system fails safely. When the HR feed stops, the system must raise an alert within an hour and must not treat missing staff as leavers and disable everyone. When a connector fails, failed operations must be retried and alerted. When a signing certificate is near expiry, alerts must fire at ninety, thirty and seven days. Sana designs each test with a synthetic trigger and an expected result, because failure handling that has never been tested usually does not work when needed.
40.3 Regression tests
Every failure from the first round becomes a permanent regression test. The wrong-audience token test runs against every integrated application, not only the one that failed, because the same mistake could exist elsewhere. The mover with an overlapping exception is turned into a family of tests covering different exception timings. Regression tests run automatically on each change, so a fix cannot silently be undone months later by an unrelated update to an application or policy. The regression suite grows with every fixed failure.
40.4 Final evidence
The second round runs 148 tests, and 146 pass. One failure is an alert that fires at thirty days but not at ninety; the other is an application that accepts a token two minutes after expiry because of clock tolerance set too generously. Both are fixed and retested. Evidence for every test is stored with its configuration version and linked from the traceability matrix. Sana summarises results by requirement, listing any residual risks accepted by the council, ready for the evidence dossier.
Worked example
The second round passes 146 of 148 tests; the two failures, a missing ninety-day certificate alert and a too-generous clock tolerance, are fixed and retested.
Synthetic case. Tests, failures and versions are synthetic.
Errors and limits
Failure handling that has never been tested usually fails when it is needed.
Exercises
- EX118 — Feed failure rule. What must not happen when the HR feed stops? Rubric: 3 = treating missing staff as leavers and disabling everyone;; 2 = mass disable;; 1 = alert;; 0 = none.
- EX119 — Second round result. How many tests passed in the second round? Rubric: 3 = 146 of 148;; 2 = 146;; 1 = 148;; 0 = none.
- EX120 — Clock tolerance. What did the clock tolerance failure allow? Rubric: 3 = tokens accepted two minutes after expiry;; 2 = late tokens;; 1 = vague;; 0 = none.
Synthesis
Broader verification with failure paths and permanent regression tests produces final evidence the council can rely on.
Continue studying — chapter 40 of 50
← Chapter 39 (draft) · Course chapter index · Chapter 41 (draft) →