Chapter 38 — Systems, endpoints and hardware security: validation
Lesson map
Lesson 38 published. Map, example, EX112.
flowchart TD
N0["Learning outcomes"]
N1["38.1 Validation"]
N2["Lab — Lab — traceability"]
N3["38.2 Reading the result"]
N4["38.3 Closing"]
N5["38.4 User validation"]
N6["Verification versus validation"]
N7["Evidence links"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX112"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["38.1 Validation"]
N2["Lab — Lab — traceability"]
N3["38.2 Reading the result"]
N4["38.3 Closing"]
N5["38.4 User validation"]
N6["Verification versus validation"]
N7["Evidence links"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX112"]
[Learning outcomes]
|
v
[38.1 Validation]
|
v
[Lab — Lab — traceability]
|
v
[38.2 Reading the result]
|
v
[38.3 Closing]
|
v
[38.4 User validation]
|
v
[Verification versus validation]
|
v
[Evidence links]
|
v
[Worked example] --> [Exercise EX112]
Learning outcomes
You will (1) validate that requirements are met in operation; (2) trace requirements to evidence; (3) identify gaps.
38.1 Validation
Validation asks whether the system meets user needs in real use; verification asked whether it met requirements in tests.
Verification used lab PCs, synthetic sessions and written tests. Validation looks at the branches in normal use: do patrons find PCs ready, do they lose work they expected to keep, do librarians spend less time on PC problems? These questions can only be answered by watching the system in service and asking the people who use it. Some answers will show needs the requirements missed.
Lab — Lab — traceability
"""Requirements traceability: which requirements have passing verification evidence (synthetic)."""
reqs = {"R1 secure boot": "pass", "R2 firmware password": "pass", "R3 no USB storage": "pass", "R4 nightly restore": "pass",
"R5 app control enforce": "pass", "R6 auto logoff": "pass", "R7 remote logging": "fail", "R8 staff disk encryption": "pass",
"R9 restore alert": "untested", "R10 accessibility tools": "pass"}
c = {s: sum(v == s for v in reqs.values()) for s in ("pass", "fail", "untested")}
print(f"requirements {len(reqs)}: pass {c['pass']}, fail {c['fail']}, untested {c['untested']}")
for r, v in reqs.items():
if v != "pass": print(f" {r}: {v}")
Run with python3 lab_trace.py. Verified output:
requirements 10: pass 8, fail 1, untested 1
R7 remote logging: fail
R9 restore alert: untested
38.2 Reading the result
Of 10 requirements, 8 pass, 1 fails (R7 remote logging) and 1 is untested (R9 restore alert in production).
The lab output gives the same summary in its first line and lists the two items that need attention. The remote logging failure is the firewall rule found during the first verification lab, not yet fixed on the PC the table recorded. The restore alert had been tested on lab PCs but not yet in production, so it is marked untested here rather than counted as a pass.
38.3 Closing
R7's firewall fix is applied and re-tested; R9 is tested by deliberately failing a lab PC's restore and checking the alert.
Deliberately failing a restore on a lab PC connected to the production management server tests the full path: the failure is detected, the alert is raised and the technician receives it. Mateo records the time from failure to alert. After both fixes, the table is rerun and the worked example shows the result. Each fix is recorded with its evidence.
38.4 User validation
Librarians report fewer patron complaints and no leftover files found in a month.
No leftover files in a month is the outcome the project was started for. Mateo is careful about what it shows: spot checks find files only on the PCs checked, and a month is a short period. He reports it as a strong early sign, not proof, and plans to continue spot checks for the rest of the year. Complaint numbers are compared with the baseline month.
Verification versus validation
Verification asks: did we build it right? Validation asks: did we build the right thing? A PC could pass every requirement test yet frustrate patrons, for example if the logoff timer were too short for people filling in long forms. Validation therefore includes librarian feedback and patron comments collected at the desk.
One comment asks for longer idle time for job applications. After discussion, auto-logoff stays at 15 minutes but shows a warning two minutes before, with a button to continue.
Evidence links
Each requirement in the table links to its evidence file.
Closing gaps
Every failed or untested item gets an owner and date.
Patron comments
Comment cards at each branch ask patrons whether PCs worked and whether anything was confusing.
Acting on feedback
The two-minute warning is tested with librarians, who confirm patrons understand it.
Final table
All requirements are linked to passing evidence after fixes.
Validation report
A short report summarises validation for Marta.
Ongoing validation
Comment cards continue after the project.
Worked example
After fixes, the traceability table shows 10 of 10 passing, with evidence links.
Synthetic case. Synthetic.
Errors and limits
Passing tests do not guarantee users are satisfied; ask them.
Exercises
- EX112 — Lab status. Give the traceability counts. Rubric: 3 = 8 pass, 1 fail, 1 untested; 2 = partial; 1 = vague; 0 = none.
- EX113 — Failing items. Which requirements failed or were untested? Rubric: 3 = R7, R9; 2 = partial; 1 = vague; 0 = none.
- EX114 — User evidence. What did librarians report? Rubric: 3 = fewer complaints, no leftover files; 2 = partial; 1 = vague; 0 = none.
Synthesis
Validate in operation and trace every requirement to evidence.
Continue studying — chapter 38 of 50
← Chapter 37 (draft) · Course chapter index · Chapter 39 (draft) →