Chapter 28 — Security architecture and engineering: failure analysis

Lesson map

Lesson 28 published. Map, example, EX082.

flowchart TD
  N0["Learning outcomes"]
  N1["28.1 The incident"]
  N2["28.2 Layers of cause"]
  N3["28.3 Lasting improvements"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX082"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["28.1 The incident"]
  N2["28.2 Layers of cause"]
  N3["28.3 Lasting improvements"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX082"]

[Learning outcomes]
    |
    v
[28.1 The incident]
    |
    v
[28.2 Layers of cause]
    |
    v
[28.3 Lasting improvements]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX082]

Learning outcomes

You will (1) run a blameless review of a Fernway incident; (2) separate causes into design, process and human factors; (3) produce improvements that last.

28.1 The incident

During a weekend sale, Fernway's shop showed some items at a price of zero for forty minutes, and twenty-three orders were placed. Hannah asks Lucas to help run a blameless review. They build a timeline from logs and interviews: a buyer uploaded a price spreadsheet with a blank column, the import treated blanks as zero, and no alert fired because the monitoring watched availability, not prices. Building the timeline before discussing causes keeps the review grounded in what actually happened rather than early assumptions.

28.2 Layers of cause

Lucas sorts contributing causes into layers. Design: the import accepted blanks as zero and the spreadsheet upload crossed a trust boundary without validation, as noted in chapter 21. Process: no second person checked large price changes before weekend sales. Human: the buyer was rushing to meet a deadline set by marketing. He avoids naming the buyer as the cause, since anyone under the same pressure with the same tools could have made the error. Blameless framing encourages people to report problems early.

28.3 Lasting improvements

The review produces four improvements: the import rejects blanks and any price change over fifty percent without approval; a dashboard alert fires when many prices change at once; a second-person check is added for sale preparation; and deadlines for sale pricing move one day earlier. Each has an owner and a date. Fernway honours the orders as goodwill. Three months later, Lucas checks the improvements are in place and finds the alert has already caught a mistaken currency conversion before customers saw it.

Worked example

Lucas helps run a blameless review of a zero-price incident, separates design, process and human causes and verifies four improvements later.

Synthetic case. The incident and figures are synthetic.

Errors and limits

Blameless does not mean accountless; owners are still named for fixes. Timelines from memory need log confirmation.

Exercises

• EX082 — Why no alert. Why did no alert fire?
  Show marking criteria

  Rubric: 3 = monitoring watched availability, not prices;; 2 = monitoring gap;; 1 = vague;; 0 = none.
• EX083 — Not blaming the buyer. Why avoid naming the buyer as the cause?
  Show marking criteria

  Rubric: 3 = anyone under the same pressure and tools could err;; 2 = system;; 1 = blame;; 0 = none.
• EX084 — Later proof. What showed the improvements worked?
  Show marking criteria

  Rubric: 3 = the alert caught a currency conversion mistake;; 2 = alert;; 1 = vague;; 0 = none.

Synthesis

Blameless failure analysis finds design and process causes and produces improvements that can be checked later.

Continue studying — chapter 28 of 40

← Chapter 27 (draft) · Course chapter index · Chapter 29 (draft) →