Chapter 22 — Automate repeatable evidence and notification tasks
Lesson map
Lesson 22 published. Map, example, EX064.
flowchart TD
N0["Learning outcomes and connection"]
N1["22.1 What to automate"]
N2["22.2 Design rules"]
N3["22.3 Lab T-22 synthetic : evidence manifest with ver"]
N4["22.4 Notifications"]
N5["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX064"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["22.1 What to automate"]
N2["22.2 Design rules"]
N3["22.3 Lab T-22 synthetic : evidence manifest with ver"]
N4["22.4 Notifications"]
N5["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX064"]
[Learning outcomes and connection]
|
v
[22.1 What to automate]
|
v
[22.2 Design rules]
|
v
[22.3 Lab T-22 synthetic : evidence manifest with ver]
|
v
[22.4 Notifications]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX064]
Learning outcomes and connection
You will be able to (1) choose which response tasks to automate and which to keep human; (2) build evidence-collection automation that preserves integrity; and (3) automate internal notifications with safeguards against error. Chapter 3 introduced scripting; Chapters 17–21 provided the tools and processes into which automation now plugs.
22.1 What to automate
Good candidates are frequent, well-defined and low-judgement: enriching an alert with inventory and identity data; exporting standard log sets for a case; hashing and registering artefacts; creating linked records; paging; reminding owners of overdue tasks; drafting status updates from case fields. Keep humans for judgement and irreversible or high-impact actions: severity decisions beyond rules, external notifications, disabling public services, contacting employees. CISA's playbooks and many security-orchestration tools follow the same split between automated enrichment and human-approved response. Fernmoor's enrichment that paged a former employee shows the risk of trusting automated data, even for low-judgement tasks.
22.2 Design rules
Each automation has an owner, a version under source control, tests with known inputs, logging of every run (inputs, outputs, errors), idempotence (running twice does no harm), safe failure (on error, stop and alert a human rather than half-complete), and least-privilege credentials stored in a secrets manager, not in the script. Fernmoor's enrichment failure would have been caught earlier by these rules. A test with a known stale owner would have exposed the missing validation, and logging each run would have shown three weeks of pages to the same personal number. The fix added an owner check against the directory and a fallback to the team rota.
22.3 Lab T-22 (synthetic): evidence manifest with verification
This script builds a SHA-256 manifest for three exported files of case FCS-214, then simulates a later verification in which one file was re-saved by a text editor that added a newline. It prints the manifest, the failure and a structured note for the case.
# Lab T-22: build and verify an evidence manifest (SHA-256, FIPS 180-4)
import hashlib, json
items = {"idp_export_FCS-214.json": b'{"user":"permits-clerk-4","events":12}\n',
"vpn_FCS-214.log": b"09:03:02 session start\n09:41:30 session end\n",
"fs_audit_FCS-214.csv": b"time,user,files\n14:12:35,permits-clerk-4,412\n"}
manifest = {n: hashlib.sha256(b).hexdigest() for n,b in sorted(items.items())}
for n,h in manifest.items(): print(h[:16], len(items[n]), n)
# later verification: one file was re-saved by an editor that added a trailing newline
items["vpn_FCS-214.log"] += b"\n"
bad = [n for n,b in sorted(items.items()) if hashlib.sha256(b).hexdigest()!=manifest[n]]
print("verification failures:", bad)
note = {"case":"FCS-214","failed":bad,"action":"use preserved original; record deviation"}
print(json.dumps(note, sort_keys=True))
Expected output:
bdeae596d177e5ad 45 fs_audit_FCS-214.csv
ae76b3b651ddaa47 39 idp_export_FCS-214.json
847409adc14e0145 44 vpn_FCS-214.log
verification failures: ['vpn_FCS-214.log']
{"action": "use preserved original; record deviation", "case": "FCS-214", "failed": ["vpn_FCS-214.log"]}
The manifest shows the first 16 hex characters of each hash and the byte length (store the full hash in practice). A single added byte changes the hash completely, so verification detects the edit; the response is to use the preserved original from the write-once store and record the deviation, not to "fix" the working copy. The structured note is suitable for the case API.
22.4 Notifications
Internal notifications (page the on-call, inform the privacy officer at S2 with personal data, update the service desk) can be automated from case fields. Safeguards: templates reviewed by the owners; recipients from a maintained directory, not typed; rate limits; a test mode; and a human check for any message leaving Fernmoor. External notifications to residents, regulators or the press are drafted by automation at most, approved and sent by people authorized by the charter (Chapter 10). In the ransomware outage, the internal page and service desk updates were automated, while the resident notice was drafted for the director.
22.5 Worked example (synthetic): enrichment that misled
Fictional. An enrichment step added the inventory owner to each alert. A stale CMDB entry named a person who had left Fernmoor; the automation paged her personal phone for three weeks. Fixes: the automation now validates owners against the active directory and falls back to the team rota; the inventory reconciliation (Chapter 11) flags owners who left. The error lasted three weeks because nothing checked whether the person paged still worked at Fernmoor, and the pages went to a phone that nobody at Fernmoor could see. The fix follows the design rules: validation against an authoritative source, safe fallback to a rota, and a reconciliation that catches the stale entry at its source. The case was also logged as an automation defect, so the incident review covered the design and not only the CMDB record.
22.6 Common errors and limits
- Automating a broken process, which produces wrong results faster.
- Credentials embedded in scripts.
- Silent partial failures.
- Automations nobody owns after their author moves on.
The stale-owner enrichment shows the first and fourth errors: the process of keeping owners current was already broken, and the automation multiplied its effect, while nobody noticed because the automation's author had moved to another project. Fernmoor now lists an owner for every automation in its register, stores credentials in the secrets manager, and makes partial failures stop and alert.
Exercises — chapter 22
Attempt each exercise before opening the worked results. Every case, organization, person, address and number in an exercise is synthetic unless a cited public source is named.
EX064 — Automate or not. Steps: classify eight tasks as automate, automate-with-approval or human: enrich alert; disable user; isolate workstation; notify regulator; hash artefacts; create IT incident link; email all staff about phishing; page on-call. Rubric: 3 = all defensible with reasons; 2 = six; 1 = three; 0 = none.
EX065 — Extend the lab. Steps: modify the lab to store and print the full SHA-256 and to report files added or missing since the manifest. Rubric: 3 = code reports changed, added and missing; full hash; tested by adding a file; 2 = one of added/missing; 1 = attempted; 0 = none.
EX066 — Run log. Steps: define the fields an automation run log must record. Rubric: 3 = automation id/version, trigger, case id, inputs (with hashes), outputs, errors, start/end time, credential identity; 2 = five; 1 = two; 0 = none.
Chapter synthesis
Automate frequent, well-defined, low-judgement tasks with owners, tests, logs, idempotence, safe failure and least privilege; verify evidence by hash against preserved originals; and keep humans in charge of judgement and anything leaving the organization.
Continue studying — chapter 22 of 30
← Chapter 21 · Course chapter index · Chapter 23 →
Continue to the next authored lesson.