◇ OpenCyber Academy
Engineering · Chapter 40 · English

Chapter 40 — Verification laboratory II

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 40 — Verification laboratory II

Lesson map

Lesson 40 published. Map, example, EX118.

flowchart TD
  N0["Learning outcomes"]
  N1["40.1 Regression verification"]
  N2["40.2 Load verification"]
  N3["Lab — Verdict times during a synthetic campaign day"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX118"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["40.1 Regression verification"]
  N2["40.2 Load verification"]
  N3["Lab — Verdict times during a synthetic campaign day"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX118"]

[Learning outcomes]
    |
    v
[40.1 Regression verification]
    |
    v
[40.2 Load verification]
    |
    v
[Lab — Verdict times during a synthetic campaign day]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX118]

Learning outcomes

You will (1) re-verify after changes; (2) verify controls under realistic load; (3) report verification results with limits.

40.1 Regression verification

After a year of changes, including new guidance, automatic verdicts and the vendor gate, Aiko reruns the verification table from chapter 30 and adds tests for new requirements. Regression verification checks that changes have not broken something that worked before. One regression appears: the automatic verdict feature sends feedback without the link to guidance added after the usability test. The IT team restores the link and the retest passes. Without the table, the regression might have gone unnoticed for months while staff lost useful guidance.

40.2 Load verification

Aiko verifies the triage service under realistic load by replaying a synthetic campaign day with three hundred reports in two hours. She measures the time from report to verdict with automatic clustering and verdicts enabled. The lab summarises the result: most reports receive verdicts within minutes because they cluster into a few distinct emails, but a handful of unique reports wait longer for an analyst. The service level of one hour for ninety percent is met, and the slowest report waits under two hours.

Lab — Verdict times during a synthetic campaign day

# Verdict times for a synthetic campaign day: 300 reports in 2 hours.
import random
r = random.Random(7)
times = []
for i in range(300):
    if i < 270: times.append(r.uniform(1, 12))      # clustered, automatic verdicts
    elif i < 273: times.append(r.uniform(61, 104))  # unique reports waiting for analyst
    else: times.append(r.uniform(15, 58))           # unique reports handled within the hour
times[-1] = 104.0
within = sum(t <= 60 for t in times) / len(times)
print(f"within 60 min: {within:.0%}; slowest {max(times):.0f} min")

Run with python3 lab_load.py. Verified output:

within 60 min: 99%; slowest 104 min

40.3 Reporting limits

Aiko reports the results with limits. The replay used synthetic reports and could not include the confusion of a real campaign, such as staff calling the help desk. Load tests ran outside working hours, so they did not compete with other systems. She recommends a live check during the next real campaign. The verification report states clearly what was tested, what was not and what remains uncertain, which allows the risk committee to judge how much confidence to place in the service.

Worked example

Aiko reruns verification and finds a missing link regression, load-tests triage with a synthetic campaign and reports limits openly.

Synthetic case. Tests are synthetic.

Errors and limits

Synthetic replays miss real confusion. Live checks are harder to control.

Exercises

Synthesis

Re-verification catches regressions, load tests check scale, and honest limits guide confidence.

Continue studying — chapter 40 of 50

← Chapter 39 (draft) · Course chapter index · Chapter 41 (draft) →