Chapter 45 — Independent design review | Engineering Digital forensics and malware analysis

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 45 — Independent design review

Lesson map

Lesson 45 published. Map, example, EX133.

flowchart TD
  N0["Learning outcomes"]
  N1["45.1 Purpose"]
  N2["45.2 Review package"]
  N3["45.3 Findings"]
  N4["45.4 Response"]
  N5["45.5 Evidence of closure"]
  N6["45.6 Why the major finding happened"]
  N7["45.7 Process lesson"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX133"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["45.1 Purpose"]
  N2["45.2 Review package"]
  N3["45.3 Findings"]
  N4["45.4 Response"]
  N5["45.5 Evidence of closure"]
  N6["45.6 Why the major finding happened"]
  N7["45.7 Process lesson"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX133"]

[Learning outcomes]
    |
    v
[45.1 Purpose]
    |
    v
[45.2 Review package]
    |
    v
[45.3 Findings]
    |
    v
[45.4 Response]
    |
    v
[45.5 Evidence of closure]
    |
    v
[45.6 Why the major finding happened]
    |
    v
[45.7 Process lesson]
    |
    v
[Worked example] --> [Exercise EX133]

Learning outcomes

You will (1) prepare EvidenceDesk for independent review; (2) respond to review findings; (3) distinguish findings by severity; (4) close findings with evidence.

45.1 Purpose

An independent reviewer — an external forensic engineer unconnected to the project — examines design, code, validation and pilot records. Independence matters because builders overlook their own assumptions.

An unconnected reviewer has nothing to gain from approving the work and no shared history that makes some choices seem obvious. Priya and Northfield agree the reviewer's terms in writing: what they will see, how long they have, and that findings will be reported in full. The reviewer also confirms in writing that they have no conflict of interest with the project.

45.2 Review package

Requirements and traceability, architecture and decision records, threat and failure analysis, validation plan and report, verification campaigns, pilot reports, debt register.

Assembling the package is itself a check: missing or out-of-date documents become obvious when everything must be handed over at once. Priya finds that the threat analysis predates the anchoring change after the verification campaign and updates it before sending. Each document carries a version and date so the reviewer's findings can refer to exact versions. The reviewer also gets read access to the code.

45.3 Findings

The reviewer raises seven findings. Major: anchors are stored on the same server as the chain during the pilot, defeating their purpose if the server is compromised. Moderate: the developer guide lacks a release-signing step. Minor: inconsistent field names in two modules, among others.

Grading findings tells the project where to spend effort first. The anchor finding is major because it undermines tamper detection, the reason anchors exist: anyone who could edit the chain on that server could also edit the anchors alongside it. The release-signing gap is moderate because releases could be swapped unnoticed. Inconsistent names cause confusion but no direct risk.

45.4 Response

Each finding gets a response: accept and fix, accept and defer with reason, or dispute with evidence. Priya accepts all seven; the anchor issue is fixed first by moving anchors to the quality manager's separate signed log as originally designed.

Disputing a finding is legitimate when the reviewer misunderstood something, but it must rest on evidence rather than preference. Here Priya agrees with every finding. Moving anchors to the quality manager's log restores the separation in the original design: changing history would then require altering two systems controlled by different people.

45.5 Evidence of closure

Each closed finding links to the change, the test that proves it, and the reviewer's confirmation.

Saying a finding is fixed is a claim; linking the change, the test and the reviewer's confirmation makes it checkable. For the anchor finding, the test edits the chain on the server and confirms verification fails against the separate log. The reviewer reruns that test personally before marking the finding closed. Open findings stay listed on the review summary until all three links are present, so nothing is marked closed on a promise.

45.6 Why the major finding happened

During pilot setup, anchoring was configured locally "temporarily" and never moved. The debt register had not captured it.

Setting up quickly to start the pilot on time, Priya pointed anchors at a local folder intending to fix it within a week, and other tasks took over. No document recorded the shortcut, so nothing reminded anyone. The finding is less about anchors than about how temporary decisions disappear from view. Hugo points out that the same pattern appears in lab practice, such as a temporary storage shelf that becomes permanent.

45.7 Process lesson

Temporary configurations go in the debt register with a deadline.

A deadline turns 'temporarily' into a date someone is accountable for. The daily check sheet gains a line listing debt items past their deadline, so an overdue shortcut is visible every morning. Priya also adds a release criterion: no temporary configuration may be live in production without a register entry. The rule applies to lab procedures as well, so a temporary workaround for a broken write blocker, for example, would also need an entry and a date before anyone relied on it.

45.8 Communicating results

The review summary is shared with Northfield's manager, including the major finding, rather than only the closed status.

Reporting only 'all findings closed' would be accurate but would hide that a key control was ineffective for part of the pilot. The manager needs to know because pilot records from that period had weaker tamper protection. The summary states the exposure window and confirms that chain verification against the new anchors found no changes. The manager can then weigh that exposure directly.

Choosing a reviewer

Priya and Hugo pick a reviewer with forensic engineering experience who has no ties to the project or to vendors under consideration, and record that independence check.

Worked example

Priya's response table row: "F1 (major) — anchors co-located with chain. Accepted. Anchors moved to QM signed log 14 March; test T-ANCH-03 confirms chain rebuild is detected; reviewer confirmed 18 March."

Synthetic case. The review is invented.

Errors and limits

One review sees what one reviewer looks for; combine with ongoing audit.

Exercises

EX133 — Major finding. What was the major finding, and why does it matter? Rubric: 3 = anchors on same server; compromise defeats detection; 2 = partial; 1 = vague; 0 = none.
EX134 — Response options. List three ways to respond to a finding. Rubric: 3 = fix, defer with reason, dispute with evidence; 2 = two; 1 = one; 0 = none.
EX135 — Root cause. Why did the major issue occur, and what process change followed? Rubric: 3 = temporary config not moved; debt register with deadline; 2 = one; 1 = vague; 0 = none.

Synthesis

Independent review finds what builders miss; close every finding with linked evidence and learn from its cause.

Continue studying — chapter 45 of 50

← Chapter 44 (draft) · Course chapter index · Chapter 46 (draft) →