Chapter 38 — Security architecture and engineering: validation
Lesson map
Lesson 38 published. Map, example, EX112.
flowchart TD
N0["Learning outcomes"]
N1["38.1 Building the right system"]
N2["38.2 Red-team findings"]
N3["38.3 User studies"]
N4["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4
N4 --> EX["Worked example"]
EX --> Q["Exercise EX112"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["38.1 Building the right system"]
N2["38.2 Red-team findings"]
N3["38.3 User studies"]
N4["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX112"]
[Learning outcomes]
|
v
[38.1 Building the right system]
|
v
[38.2 Red-team findings]
|
v
[38.3 User studies]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX112]
Learning outcomes
You will (1) distinguish validation from verification; (2) validate the architecture against stakeholder needs and real use; (3) use red-team exercises and user studies as validation evidence.
38.1 Building the right system
Verification asked whether the system was built as specified; validation asks whether the specification meets real needs. A perfectly verified system can still fail its users or miss real threats. Sofia plans validation for the back office: review with stakeholders whether privacy needs are actually met in daily practice, observe customer service agents using the dispute screens and commission an authorised red-team exercise to test whether realistic attackers can achieve their goals despite the controls. Validation often finds problems in requirements themselves, which no amount of verification would reveal.
38.2 Red-team findings
An authorised red team from a specialist firm, working under written rules of engagement that exclude live passenger data and vehicles, attempts to obtain journey histories. They do not break any cryptography; instead, they phish a customer service agent and use the agent's legitimate access to look up journeys of chosen people. Every control worked as specified. The finding shows a requirement gap: agents could look up any card without a linked dispute. Sofia adds a requirement that lookups need an open case and are reviewed, which verification would never have found.
38.3 User studies
Sofia also observes six agents handling synthetic disputes. They find the new screens confusing and share notes in a chat tool, copying journey details outside the protected system. This is a validation failure of psychological acceptability. The team redesigns the screens to show case notes inside the system and removes the need to copy data. Sofia notes that security controls that ignore real work patterns push people into workarounds that undermine the architecture, so user studies are a security activity, not only a usability one.
Worked example
Sofia plans validation, a red team obtains journey data via a phished agent's legitimate access revealing a requirement gap, and a user study finds data copied to chat, leading to redesigned screens.
Synthetic case. The red team, agents and findings are synthetic.
Errors and limits
Red-team exercises cover only the paths tried. Observed users may behave differently when watched.
Exercises
- EX112 — Validation versus verification. How does validation differ from verification? Rubric: 3 = validation checks the specification meets real needs; verification checks build matches specification;; 2 = both;; 1 = same;; 0 = none.
- EX113 — Red-team route. How did the red team obtain journey histories? Rubric: 3 = phished an agent and used legitimate access to look up chosen cards;; 2 = phishing;; 1 = broke crypto;; 0 = none.
- EX114 — Workaround cause. Why did agents copy data to chat? Rubric: 3 = screens were confusing and lacked case notes;; 2 = usability;; 1 = malice;; 0 = none.
Synthesis
Validation tests whether the architecture serves real needs and resists real attackers, revealing requirement gaps verification cannot.
Continue studying — chapter 38 of 50
← Chapter 37 (draft) · Course chapter index · Chapter 39 (draft) →