Chapter 14 — Human factors and cyber policy: automation
Lesson map
Lesson 14 published. Map, example, EX053.
flowchart TD
N0["Learning outcomes"]
N1["14.1 Automating triage"]
N2["Lab — Reports needing human review after automation"]
N3["14.2 Human review"]
N4["14.3 Monitoring automation"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX053"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["14.1 Automating triage"]
N2["Lab — Reports needing human review after automation"]
N3["14.2 Human review"]
N4["14.3 Monitoring automation"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX053"]
[Learning outcomes]
|
v
[14.1 Automating triage]
|
v
[Lab — Reports needing human review after automation]
|
v
[14.2 Human review]
|
v
[14.3 Monitoring automation]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX053]
Learning outcomes
You will (1) automate triage steps for reports; (2) keep human review where needed; (3) monitor automated steps.
14.1 Automating triage
Priya automates routine triage. Reports about the same message are grouped. A thank-you note is sent immediately. If a reported message matches one already confirmed as malicious, the reporter receives the standard verdict and the message is removed from other mailboxes automatically. The IT lead reviews only new messages. The lab counts how many reports in a synthetic busy week needed human review after automation, showing how much time is saved for replies and investigation. The rules are simple to read and change.
Lab — Reports needing human review after automation
# Reports needing human review after automation in a synthetic busy week.
total, dup, known = 80, 51, 17
print(f"total {total}; duplicates {dup}; known verdicts {known}; human review {total - dup - known} ({(total - dup - known) / total:.0%})")
Run with python3 lab_triage.py. Verified output:
total 80; duplicates 51; known verdicts 17; human review 12 (15%)
14.2 Human review
Priya considers whether to let a scoring service decide verdicts for all messages. She decides against it, because wrongly labelling a real customer email as malicious could cause a missed delivery or a lost customer, and wrongly labelling a fake as safe could cause fraud. Human review stays for new messages. She notes that automation should handle what is certain and repetitive, while people handle what is uncertain or has serious consequences. The IT lead agrees with this division of work and records it in the triage runbook.
14.3 Monitoring automation
Automated steps can fail without anyone noticing, as the South depot incident showed. Priya adds a daily log line for each automated step and a weekly summary that lists failures. She also samples ten automated verdicts each month for the IT lead to check by hand. In the first month, all ten are correct. Monitoring gives the team confidence in automation and catches errors before they harm trust in the reporting service. The summary arrives every Monday morning by email. Results are logged.
Worked example
Priya automates grouping, thanks and known verdicts, keeps human review for new messages and monitors automation with logs and samples.
Synthetic case. Data are synthetic.
Errors and limits
Known lists need updates. Samples may miss rare errors.
Exercises
- EX053 — Lab review share. How many of 80 reports needed human review in the lab? Rubric: 3 = 12;; 2 = 15%;; 1 = 80;; 0 = none.
- EX054 — Scoring service. Why not let a scoring service decide all verdicts? Rubric: 3 = mislabelling causes lost customers or fraud;; 2 = consequences;; 1 = cost;; 0 = none.
- EX055 — Automation rule. What should automation handle? Rubric: 3 = what is certain and repetitive;; 2 = routine;; 1 = judgement;; 0 = none.
- EX056 — Monthly sample. What monthly check did Priya add? Rubric: 3 = hand-checking ten automated verdicts;; 2 = sample;; 1 = vague;; 0 = none.
Synthesis
Automation speeds feedback when it handles certainty and leaves judgement to people.
Continue studying — chapter 14 of 18
← Chapter 13 (draft) · Course chapter index · Chapter 15 (draft) →