◇ OpenCyber Academy
Engineering · Chapter 30 · English

Chapter 30 — Verification laboratory I

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 30 — Verification laboratory I

Lesson map

Lesson 30 published. Map, example, EX088.

flowchart TD
  N0["Learning outcomes"]
  N1["30.1 Lab setup"]
  N2["30.2 Test cases"]
  N3["30.3 Evidence"]
  N4["30.4 Results"]
  N5["Test design"]
  N6["Negative and positive tests"]
  N7["Regression testing"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX088"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["30.1 Lab setup"]
  N2["30.2 Test cases"]
  N3["30.3 Evidence"]
  N4["30.4 Results"]
  N5["Test design"]
  N6["Negative and positive tests"]
  N7["Regression testing"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX088"]

[Learning outcomes]
    |
    v
[30.1 Lab setup]
    |
    v
[30.2 Test cases]
    |
    v
[30.3 Evidence]
    |
    v
[30.4 Results]
    |
    v
[Test design]
    |
    v
[Negative and positive tests]
    |
    v
[Regression testing]
    |
    v
[Worked example] --> [Exercise EX088]

Learning outcomes

You will (1) verify requirements in the lab; (2) record evidence; (3) trace results.

30.1 Lab setup

The lab replicates one office with spare equipment and virtual machines representing zones.

The verification lab reproduces the design on a small scale, using spare or virtual equipment, synthetic addresses and synthetic traffic. It is isolated from the production network so tests cannot disrupt services. Its configuration is drawn from the same version-controlled files as production, so tests exercise exactly what will be deployed. Documenting the lab setup allows others to rebuild it and repeat the tests.

The lab is reset before each test run.

30.2 Test cases

Tests check R1, R2, R3 and R7 by attempting permitted and forbidden connections from test machines within the lab only.

Each test case states its purpose, the requirement it verifies, the input such as a synthetic connection attempt, and the expected result such as allowed or denied. Cases cover both needed flows that must work and forbidden flows that must fail, plus edge cases such as boundary addresses. Writing expected results before running tests prevents the engineer from reinterpreting outcomes afterwards.

30.3 Evidence

Each test records date, configuration version, expected and actual result.

Evidence records what each test actually did and showed: the date, configuration version, test input, observed output, and pass or fail. Logs or screenshots are stored with the record. Good evidence lets a reviewer confirm the result without rerunning the test, and supports the thesis and the council's acceptance. Evidence that cannot be traced to a specific configuration version is much weaker, because it may describe a setup that no longer exists.

30.4 Results

All tests pass; one test reveals a rule allowing management to cameras on an unnecessary port, which is narrowed.

Results are summarised as pass rates per requirement, with every failure listed, its cause, and the fix applied and retested. A clean summary hides nothing, because reviewers trust a report that shows failures honestly more than one where everything passed first time. Remaining failures that are accepted are recorded as residual risk with an owner.

Results are shared with the council in plain terms.

Test design

Kofi designs tests from the requirements, not from the configuration. For R1, tests attempt connections from a guest test machine to each internal zone on common services and expect every one to fail; they also confirm guest can still reach the internet, so the test detects over-blocking as well as under-blocking. Designing from requirements means the tests would catch a configuration that looks tidy but does not meet the need. Each test names the requirement it supports, which later feeds directly into the traceability table used for validation.

Negative and positive tests

Every requirement gets both positive tests, showing permitted flows work, and negative tests, showing forbidden flows fail. A policy that blocks everything would pass all negative tests, so positive tests are essential. Kofi balances the two, with roughly equal numbers. He also includes boundary tests, such as the port just beside a permitted one, to show rules are exactly as narrow as intended. Tests use synthetic devices in the lab, so no real user traffic or council data is involved in testing.

Regression testing

The lab tests are kept as a regression suite. After any policy change, the full suite runs in the lab before the change reaches production. This catches unintended side effects, such as a new rule placed above an existing deny that widens access. The suite takes about ten minutes to run and becomes part of the council's change process after the project. The suite is updated whenever a requirement or needed flow changes, so it always reflects the current agreed design. New tests are reviewed like any other change.

Worked example

The test log has 24 entries, all passing after the fix.

Synthetic case. Lab results are synthetic.

Errors and limits

Lab differences from production must be noted.

Exercises

Synthesis

Lab verification provides evidence before production rollout.

Continue studying — chapter 30 of 50

← Chapter 29 (draft) · Course chapter index · Chapter 31 (draft) →