Chapter 38 — Network and communications security: validation | Engineering Network and communications security

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 38 published. Map, example, EX112.

flowchart TD
  N0["Learning outcomes"]
  N1["38.1 Traceability"]
  N2["Lab — Requirement traceability"]
  N3["38.2 Reading the trace"]
  N4["38.3 Closing gaps"]
  N5["38.4 User validation"]
  N6["Verification and validation compared"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX112"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["38.1 Traceability"]
  N2["Lab — Requirement traceability"]
  N3["38.2 Reading the trace"]
  N4["38.3 Closing gaps"]
  N5["38.4 User validation"]
  N6["Verification and validation compared"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX112"]

[Learning outcomes]
    |
    v
[38.1 Traceability]
    |
    v
[Lab — Requirement traceability]
    |
    v
[38.2 Reading the trace]
    |
    v
[38.3 Closing gaps]
    |
    v
[38.4 User validation]
    |
    v
[Verification and validation compared]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX112]

Learning outcomes

You will (1) trace requirements to evidence; (2) identify gaps; (3) validate with users.

38.1 Traceability

Validation begins with traceability: linking every requirement to the tests that check it and to their latest results. Kofi builds on the table started during requirements work. For each of R1 to R8, the table lists the test identifiers, whether each test passed, failed or is pending, and the date and configuration version of the result. Reading across a row shows how well one requirement is supported; scanning the whole table shows which requirements have no evidence at all. The lab that follows builds this table from synthetic records so the gaps can be counted.

Lab — Requirement traceability

"""Requirement-to-test traceability for the segmentation design (synthetic, stdlib only)."""
reqs = {"R1 guest isolated": ["T1", "T2"], "R2 default deny between zones": ["T3"], "R3 mgmt via jump host only": ["T4"],
        "R4 VPN uses IKEv2 with certificates": ["T5"], "R5 logs retained 90 days": [], "R6 failover < 60 s": ["T6"],
        "R7 CCTV reaches recorder only": ["T7"], "R8 config backed up daily": []}
passed = {"T1", "T2", "T3", "T4", "T5", "T7"}
for r, ts in reqs.items():
    st = "untested" if not ts else ("pass" if all(t in passed for t in ts) else "fail/pending")
    print(f"{r:38} {','.join(ts) or '-':6} {st}")
print("covered", sum(1 for t in reqs.values() if t), "of", len(reqs))

Run with python3 lab_trace.py. Verified output:

R1 guest isolated                      T1,T2  pass
R2 default deny between zones          T3     pass
R3 mgmt via jump host only             T4     pass
R4 VPN uses IKEv2 with certificates    T5     pass
R5 logs retained 90 days               -      untested
R6 failover < 60 s                     T6     fail/pending
R7 CCTV reaches recorder only          T7     pass
R8 config backed up daily              -      untested
covered 6 of 8

38.2 Reading the trace

The lab reports that six of eight requirements have tests and five of those pass. R6, failover within sixty seconds, is pending because the production failover test has not yet been scheduled. R5, keeping logs for ninety days, and R8, daily configuration backup, have no tests at all. These gaps are easy to miss because they describe operational behaviour rather than traffic filtering, and test design naturally focuses on filtering. That is exactly why a trace is valuable: it checks coverage against the list of requirements rather than against what engineers happened to think of testing.

38.3 Closing gaps

Kofi adds two tests. For R5, an inspection test checks that the oldest record in the central log store is at least ninety days old and that the deletion job removes nothing younger. For R8, a test restores the previous day's configuration backup onto the lab firewall and confirms that it loads and passes the regression suite. Both pass. The production failover test for R6 is then scheduled with the council in an approved window and completes in forty-two seconds, inside the limit. Every requirement now has passing evidence with a date.

38.4 User validation

Passing tests shows the requirements are met, but not that the requirements captured what people actually need. To check that, Kofi asks each office manager to complete a short form after migration: was any work blocked, slowed or made harder, and how was it resolved? Two issues are reported, a scanner that could not send to a shared folder and a supplier portal that failed, and both had already been found and fixed through deny analysis. This agreement between user reports and technical monitoring gives confidence that nothing important was missed.

Verification and validation compared

Verification asks whether the system was built right: does each control behave as the requirements state? Validation asks whether the right system was built: does it meet the real needs of the people who depend on it? Kofi needs both. A network could pass every requirement test yet still block a service nobody remembered to list, or the requirements could miss a need entirely. The traceability table supports verification, while the managers' form and service desk trends support validation. Presenting both in the dossier shows examiners that the design is correct and also useful to the council in practice.

Worked example

After closing gaps, all eight requirements have passing evidence, and the trace is attached to the evidence dossier.

Synthetic case. Data are synthetic.

Errors and limits

Untested operational requirements often fail silently until needed in an incident.

Exercises

EX112 — Coverage figure. What coverage does the lab report before fixes?
Show marking criteria
Rubric: 3 = 6 of 8 covered, R5 and R8 untested, R6 pending;; 2 = 6 of 8;; 1 = vague;; 0 = none.
EX113 — R8 test. Describe a test for daily configuration backups.
Show marking criteria
Rubric: 3 = restore yesterday's backup to lab;; 2 = restore;; 1 = vague;; 0 = none.
EX114 — Validation vs verification. How does the managers' form differ from tests?
Show marking criteria
Rubric: 3 = validates needs, not requirements;; 2 = users;; 1 = vague;; 0 = none.

Synthesis

Traceability exposes gaps; closing them gives complete evidence that the design meets its requirements.

Continue studying — chapter 38 of 50

← Chapter 37 (draft) · Course chapter index · Chapter 39 (draft) →