Chapter 45 — Independent design review
Lesson map
Lesson 45 published. Map, example, EX133.
flowchart TD
N0["Learning outcomes"]
N1["45.1 Preparing for review"]
N2["45.2 Reviewer findings"]
N3["Lab — Tally of review findings by severity and statu"]
N4["45.3 Recording residual risk"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX133"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["45.1 Preparing for review"]
N2["45.2 Reviewer findings"]
N3["Lab — Tally of review findings by severity and statu"]
N4["45.3 Recording residual risk"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX133"]
[Learning outcomes]
|
v
[45.1 Preparing for review]
|
v
[45.2 Reviewer findings]
|
v
[Lab — Tally of review findings by severity and statu]
|
v
[45.3 Recording residual risk]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX133]
Learning outcomes
You will (1) prepare a design for independent review; (2) respond to reviewer findings; (3) record decisions and residual risks.
45.1 Preparing for review
Aiko submits the contractor design to an independent review panel: a security architect from another utility, a human factors researcher and a contractor representative. She provides the requirements, design, evaluation plan and known risks. Independent review brings perspectives that the designer cannot provide, because designers tend to see what they intended rather than what they built. She asks reviewers to focus on usability, fairness and whether the controls would stop the attacks identified in chapter 24. Reviewers receive the pack two weeks before the meeting.
45.2 Reviewer findings
The panel raises six findings. The architect notes that the forwarding address could be spoofed to send fake verdicts; the design is changed so verdicts arrive only through the web form account. The researcher notes that the induction module tests recall but not decisions, and suggests scenario questions. The contractor representative notes that agencies may not pass on materials, so the induction should be sent directly. The lab tallies the findings by severity and the actions taken, showing which remain open after the response.
Lab — Tally of review findings by severity and status
# Review findings tally (synthetic).
f = [("high", "closed"), ("high", "closed"), ("medium", "closed"), ("medium", "open"), ("low", "closed"), ("low", "closed")]
for s in ("high", "medium", "low"):
print(s, "closed", sum(1 for a, b in f if a == s and b == "closed"), "open", sum(1 for a, b in f if a == s and b == "open"))
print("total closed", sum(b == "closed" for _, b in f), "open", sum(b == "open" for _, b in f))
Run with python3 lab_review.py. Verified output:
high closed 2 open 0
medium closed 1 open 1
low closed 2 open 0
total closed 5 open 1
45.3 Recording residual risk
Aiko records each finding, decision and remaining risk in a review log. One medium finding remains open: contractors using personal devices may not receive verdicts quickly. She accepts this as a residual risk with a review after the pilot, and the risk owner signs the acceptance. Recording residual risks openly, in line with the risk management approach of NIST SP 800-30r1, ensures that decisions to accept risk are made by the right people and can be revisited later. The log is shared with the panel.
Worked example
Aiko submits her design to independent reviewers, addresses spoofing, induction and distribution findings, and records one accepted residual risk.
Synthetic case. The review is synthetic; NIST SP 800-30r1 is real.
Errors and limits
Reviewers may miss issues. Accepted risks may be forgotten.
Exercises
- EX133 — Spoofing finding. What did the architect find? Rubric: 3 = the forwarding address could be spoofed to send fake verdicts;; 2 = spoofing;; 1 = vague;; 0 = none.
- EX134 — Lab tally. How many findings stayed open after the response? Rubric: 3 = 5 closed, 1 medium open;; 2 = one open;; 1 = vague;; 0 = none.
- EX135 — Residual acceptance. Who signed the residual risk acceptance? Rubric: 3 = the risk owner;; 2 = owner;; 1 = the designer;; 0 = none.
Synthesis
Independent review finds what designers miss, and residual risks are recorded and owned.
Continue studying — chapter 45 of 50
← Chapter 44 (draft) · Course chapter index · Chapter 46 (draft) →