Chapter 14 — Threat intelligence and OSINT: automation
Lesson map
Lesson 14 published. Map, example, EX053.
flowchart TD
N0["Learning outcomes"]
N1["14.1 Suitable tasks"]
N2["14.2 Unsuitable tasks"]
N3["14.3 Approval gates"]
N4["14.4 Scheduling and alerts"]
N5["14.5 Documentation"]
N6["14.6 Never automated"]
N7["14.7 Trial periods"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX053"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["14.1 Suitable tasks"]
N2["14.2 Unsuitable tasks"]
N3["14.3 Approval gates"]
N4["14.4 Scheduling and alerts"]
N5["14.5 Documentation"]
N6["14.6 Never automated"]
N7["14.7 Trial periods"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX053"]
[Learning outcomes]
|
v
[14.1 Suitable tasks]
|
v
[14.2 Unsuitable tasks]
|
v
[14.3 Approval gates]
|
v
[14.4 Scheduling and alerts]
|
v
[14.5 Documentation]
|
v
[14.6 Never automated]
|
v
[14.7 Trial periods]
|
v
[Worked example] --> [Exercise EX053]
Learning outcomes
You will (1) select CTI tasks suitable for automation; (2) keep human approval for high-impact actions; (3) schedule and monitor jobs; (4) document automated behaviour.
14.1 Suitable tasks
Daily import, validation, deduplication, matching, expiry flags, weekly counts. These follow rules and repeat.
These tasks give the same result whoever performs them, so a script does them faster and without tiredness. Daily import and validation catch malformed items before they reach the platform. Weekly counts by technique feed the management summary, and the trial in the worked example showed why deduplication must be part of them. Each automated task still writes its decisions to the log, so a person can check what it did. Expiry flags only mark entries; the decision to remove or renew stays with a person.
14.2 Unsuitable tasks
Deciding relevance of new reports, grading new sources, blocking services that students depend on, sharing restricted data outward.
Each of these needs judgment about context that a rule cannot capture. Blocking a service students depend on, such as an examination board, could stop work across the college. Sharing restricted data outward cannot be undone once sent, so a person must check the label and recipient every time. Grading new sources needs knowledge of their history, which a script does not have. Relevance of new reports depends on Ashgrove's services and calendar, which change each term.
14.3 Approval gates
Automation prepares a block list; a person approves additions for the DNS filter. Low-risk categories — ESX GREEN domains graded B or better — may be auto-approved after a trial period agreed with the network team.
The approval step lets the network team see every new entry before it reaches the DNS filter. Auto-approval is limited to ESX GREEN domains graded B or better because those have the lowest risk of blocking something important. The trial period gives evidence that the category behaves as expected before anyone stops checking it.
14.4 Scheduling and alerts
Jobs run at fixed times; failure sends an alert to a shared mailbox, not one person.
Fixed times make it easy to notice when a job has not run. Sending failure alerts to a shared mailbox avoids the problem found in the system model, when ESX alerts sat unread in one person's inbox. Each alert names the job, the step that failed and where its log is stored. The shared mailbox is the one created after the ESX alert problem, read by two people. Alerts also go to the on-call person outside working hours.
14.5 Documentation
Each job: purpose, schedule, inputs, outputs, owner, alert recipient, how to disable it.
With this documentation, the on-call person can understand a job and stop it without reading the code. Inputs and outputs show what else would be affected if the job fails. Tomás keeps one document per job in the team repository, beside the job's code and tests. Tomás reviews each job document when the job changes, and the supervisor approves the update. A job without a current document is not allowed to run in production. The same template is used for every job, so readers know where to find each field.
14.6 Never automated
Contact with suspicious sites, scanning, downloading samples, posting to public channels.
These actions could alert attackers, bring malicious files onto college systems or publish restricted material, and they are forbidden even when done by hand. Automating them would also remove the human check that might catch a mistake. The rule is written on the safety sheet and in each job's documentation. The rule is checked during code review of every new job. Any code that would make network requests to indicators, rather than to ESX or college systems, is rejected. Tomás's scripts read only exported college files and ESX's TAXII collection.
14.7 Trial periods
New automation runs alongside manual work for two weeks; results are compared before switching.
Running both side by side shows any difference between the script and the manual result before the script is trusted. In the weekly counts trial, the duplicate ESX item appeared as a difference in the T1566 tally. Tomás switched to the script only after its counts matched his manual tally. The comparison is recorded day by day, so any difference can be traced. If a difference cannot be explained, the trial is extended rather than ended.
14.8 Kill switch
Every job can be disabled quickly by the on-call person without editing code.
A job that sends a wrong block list to the DNS filter must be stopped within minutes. Editing code under pressure out of hours risks further mistakes. Each job therefore has a documented switch, such as a flag in its configuration file, that the on-call person can change. The switch is tested at the end of each trial period, by the on-call person following the documented steps. Stopping a job does not remove lists already loaded, so the runbook also explains how to roll back the last change.
14.9 Review
Twice a year, jobs no longer serving a requirement are retired.
A requirement may end, for example when a supplier is replaced or ESX changes how it shares. Jobs left running after that still consume attention when they fail and may feed stale data into controls. The review checks each job against a current requirement and its rule logs before deciding to keep it. The rule logs show which rules fired and how often in the past six months. A retired job's code stays in the repository with a note explaining why it stopped.
Logging automated decisions
Every automated action writes what it received, which rule it applied and what it produced. When a lecturer asks why a site was blocked, the log answers in seconds. Logs also let the team measure how often each rule fires, which helps retire rules that never trigger and tune those that fire too often. Keep these logs for at least as long as the blocks they explain, and store them where the network and help-desk teams can read them during out-of-hours calls.
Worked example
Tomás automates weekly counts by technique. During the two-week trial his script counts one item twice because ESX re-sent it with a new identifier. He adds deduplication by indicator value and the counts match his manual tally.
Synthetic case. The duplicate is invented.
Errors and limits
Automation repeats mistakes quickly; test with tricky cases.
Exercises
- EX053 — Auto-approval. Which category may be auto-approved after a trial? Rubric: 3 = ESX GREEN domains graded B+; 2 = partial; 1 = all; 0 = none.
- EX054 — Kill switch. Why must every job have a quick disable method? Rubric: 3 = stop harm fast without code edits; 2 = partial; 1 = unnecessary; 0 = none.
- EX055 — Trial catch. What did the trial reveal about the weekly counts? Rubric: 3 = duplicate from re-sent item; dedupe by value; 2 = one; 1 = vague; 0 = none.
- EX056 — Job document. List five fields documenting a job. Rubric: 3 = five of purpose, schedule, inputs, outputs, owner, alerts, disable; 2 = three; 1 = one; 0 = none.
Synthesis
Automate rules, keep humans on judgment and high impact, monitor jobs, document them and keep a kill switch.
Continue studying — chapter 14 of 18
← Chapter 13 (draft) · Course chapter index · Chapter 15 (draft) →