◇ OpenCyber Academy
Professional · Chapter 8 · English

Chapter 8 — AI and machine-learning security: supervised case analysis

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 8 — AI and machine-learning security: supervised case analysis

Lesson map

Lesson 08 published. Map, example, EX036.

flowchart TD
  N0["Learning outcomes"]
  N1["8.1 Case analysis"]
  N2["8.2 Timeline"]
  N3["8.3 Causes and contributing factors"]
  N4["8.4 Proportionate fixes"]
  N5["8.5 Writing it up"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX036"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["8.1 Case analysis"]
  N2["8.2 Timeline"]
  N3["8.3 Causes and contributing factors"]
  N4["8.4 Proportionate fixes"]
  N5["8.5 Writing it up"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX036"]

[Learning outcomes]
    |
    v
[8.1 Case analysis]
    |
    v
[8.2 Timeline]
    |
    v
[8.3 Causes and contributing factors]
    |
    v
[8.4 Proportionate fixes]
    |
    v
[8.5 Writing it up]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX036]

Learning outcomes

You will (1) analyse an AI incident or near miss under supervision; (2) reconstruct a timeline from evidence; (3) separate causes and contributing factors; (4) recommend proportionate fixes; and (5) write a case analysis.

8.1 Case analysis

A case analysis explains what happened, why, and what should change. It is blameless: it focuses on systems and decisions, not individuals, so people report problems openly.

The FraudFlag storm case in the worked example below is written this way. A good analysis states the impact in numbers, a timeline with sources, the direct cause and contributing factors, and fixes with owners. Sofia's FraudFlag write-up names the missing link between form changes and the model, not the person who added the storm reference field. Keep the write-up short enough for managers to read in full, with detail in an appendix.

8.2 Timeline

From logs and traces: when content entered, when the model read it, what it proposed, what was executed, when it was noticed, what containment happened. Precise times help separate cause from coincidence.

For the FraudFlag storm case, the timeline begins with the new form field on Monday and ends with the fix and the alert. Record each time with its source, such as a deployment log or a dashboard, so readers can check it. Note gaps in the timeline honestly.

8.3 Causes and contributing factors

The direct cause is what produced the harm (an injected instruction led to a tool call). Contributing factors made it possible or worse (no ownership check; uploads not treated as untrusted; no alert on urgent-booking spikes).

In the FraudFlag storm case, the direct cause was the unhandled new field, and the contributing factors were the missing link between form changes and the model and the missing alert. Name each factor with its evidence, not as a guess. Ask why at each step.

8.4 Proportionate fixes

Fix the cause and the most important contributing factors. Prefer controls that bound harm. Add a regression test and, where useful, a monitoring signal.

For the FraudFlag case, the fixes were mapping the new field, adding the model to the form change checklist and alerting when the flag rate doubles. Each has an owner and a date. Avoid adding fixes that do not address a cause or factor named in the analysis. Each fix should be proportionate: cheap fixes for small risks, stronger ones for repeated or serious failures. Check after a month that each is in place.

8.5 Writing it up

Summary; timeline; impact; causes; contributing factors; actions with owners and dates; lessons. MITRE ATLAS case studies show a useful style for describing techniques.

Sofia's write-up of the storm case followed this outline, with impact quantified as 4,100 claims delayed by two days on average. Ask a colleague who was not involved to read it for clarity before it is shared. Write it within a week, while the details are fresh. Quantifying impact as Sofia did, 4,100 claims delayed by two days, gives readers a number to check rather than an adjective to argue about.

8.6 Worked example (synthetic Meridian case)

During a storm, FraudFlag flags 30% of claims instead of 6%. Timeline: a new claim form field ("storm reference") was added on Monday; FraudFlag received it as an unknown category; scores jumped. No attack. Direct cause: an unhandled new input. Contributing: no change-management link between form changes and the model; no alert on flag-rate spikes. Fixes: map the new field; add the model to the form change checklist; alert when flag rate doubles. Under supervision, Sofia writes the analysis; her supervisor asks her to quantify impact (4,100 claims delayed by an average of two days).

8.7 Errors and limits

Each error damages the analysis. Blaming individuals discourages people from reporting the next problem, which is why case analyses are blameless. Fixing only the symptom, such as remapping the field without an alert or a checklist link, leaves the next form change free to cause the same failure. A timeline without evidence turns guesses into findings. Check each case analysis for all three. Fix any of them before sharing it. Sofia's storm write-up shows how to avoid them.

Exercises

Synthesis

Supervised case analysis turns incidents into learning: evidence-based timelines, causes and contributing factors, and proportionate fixes.

Continue studying — chapter 08 of 12

← Chapter 07 (draft) · Course chapter index · Chapter 09 (draft) →