Chapter 22 — Threat intelligence and OSINT: automation | Technologist Threat intelligence and OSINT

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 22 — Threat intelligence and OSINT: automation

Lesson map

Lesson 22 published. Map, example, EX064.

flowchart TD
  N0["Learning outcomes"]
  N1["22.1 Good candidates"]
  N2["22.2 Poor candidates"]
  N3["22.3 Human checkpoints"]
  N4["22.4 Logging"]
  N5["22.5 Unsafe automation"]
  N6["22.6 Scheduling"]
  N7["22.7 Small steps"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX064"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["22.1 Good candidates"]
  N2["22.2 Poor candidates"]
  N3["22.3 Human checkpoints"]
  N4["22.4 Logging"]
  N5["22.5 Unsafe automation"]
  N6["22.6 Scheduling"]
  N7["22.7 Small steps"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX064"]

[Learning outcomes]
    |
    v
[22.1 Good candidates]
    |
    v
[22.2 Poor candidates]
    |
    v
[22.3 Human checkpoints]
    |
    v
[22.4 Logging]
    |
    v
[22.5 Unsafe automation]
    |
    v
[22.6 Scheduling]
    |
    v
[22.7 Small steps]
    |
    v
[Worked example] --> [Exercise EX064]

Learning outcomes

You will (1) choose which CTI tasks to automate; (2) design automation with human checkpoints; (3) log automated decisions; (4) avoid automating unsafe actions.

22.1 Good candidates

Repetitive, rule-based tasks: importing feeds, normalising, deduplicating, matching logs, flagging expiries, producing weekly counts.

Each of these tasks follows written rules that give the same answer every time, such as the normalisation rules in chapter 12 or the age limits in the expiry lab. Automating them saves Lena time and removes mistakes made while typing values. The worked example below shows expiry flagging catching two items Lena had missed by hand. Weekly counts for the monthly table are a further good case, since they come from logs that already exist and need no judgement to add up.

22.2 Poor candidates

Judgment tasks: deciding relevance of a novel report, grading new sources, writing assessments, blocking on critical systems.

Each of these needs judgement about context that a script does not have. Deciding whether a novel report matters to CMS depends on asset knowledge and current events, and grading a new source depends on its track record. A block on a critical system needs the owner's decision, as chapter 18 sets out. Lena keeps a short list of these judgement tasks in the pipeline documentation, so nobody automates one by accident.

22.3 Human checkpoints

Automation proposes; humans approve where impact is high. Lena's pipeline auto-exports indicators to the DNS filter only from MCX at TLP:GREEN or CLEAR with grade B or better; others wait for review.

The rule combines three conditions: MCX as source, a sharing label that allows the export, and a grade of B or better. An item meeting all three is low risk to block automatically; anything else, such as a vendor item at TLP:AMBER, waits for Lena. The checkpoint is therefore set by risk rather than applied to everything.

22.4 Logging

Every automated action logs input, rule applied and output. When a block causes complaints, the log shows exactly why it happened.

If a resident complains that a site is blocked, the log shows the input, for example a domain from MCX graded B2, the rule that exported it and when. The block record and this log together let the service desk answer quickly and lift a mistaken block. Without this logging, the automation would be a black box to everyone. The log is kept as long as the block record, so an old block can still be explained when it is reviewed months later.

22.5 Unsafe automation

Never automate contact with adversary infrastructure — automated visits, downloads or scans of suspicious sites. Never automate sharing of AMBER or RED data outward.

Automated contact with suspicious sites could alert their operators, download harmful files or breach site terms, and it would happen at machine speed. Automatic sharing of AMBER or RED data could break the conditions under which partners shared it. These two limits protect CMS and its partners whatever else is automated. Lena writes both limits at the top of the pipeline documentation, so anyone extending it sees them first.

22.6 Scheduling

Jobs run at fixed times with alerts on failure. A job that fails silently is worse than none.

A failed job with an alert is noticed and fixed; a failed job without one leaves people believing the work is done. The expired TAXII key in chapter 20 is the kind of failure that a silent job would hide. Fixed times also make it clear when results should appear, so a missing result is itself noticeable. Each job's failure alert goes to its named owner, so a failure always reaches a person who knows how to fix it.

22.7 Small steps

Automate one task, run it alongside manual work for two weeks, compare results, then switch.

Running both side by side gives a direct comparison on the same items. In the worked example, two weeks showed that the script found two items Lena had missed and missed none she had found. Only then was manual work replaced, and a monthly spot-check keeps a small human comparison going. The comparison is recorded with dates and counts, so the decision to switch can be reviewed later if the script misbehaves.

Maintaining automated jobs

Each job has an owner, a short description, its schedule and its failure alert recipient. When a job's owner changes role, ownership transfers explicitly; orphaned jobs are a frequent cause of silent pipeline decay. Twice a year Lena reviews every job and retires those that no longer serve a requirement, because each running job is code that must be maintained and secured. A job inventory makes this review quick, and it also helps auditors see exactly which decisions are taken by machines and which by people.

Worked example

Lena automates weekly expiry flagging. For two weeks she compares the script's list with her manual list: the script catches two items she missed and misses none. She switches to the script, keeping a monthly manual spot-check.

Synthetic case. The comparison is invented.

Errors and limits

Automation multiplies mistakes as fast as successes; test thoroughly.

Exercises

EX064 — Auto-export rule. Which items does Lena's pipeline auto-export to DNS? Rubric: 3 = MCX, GREEN/CLEAR, grade B or better; 2 = two conditions; 1 = all; 0 = none.
EX065 — Never automate. Name two things that must never be automated. Rubric: 3 = contact with adversary infra; outward AMBER/RED sharing; 2 = one; 1 = imports; 0 = none.
EX066 — Parallel run. Why run automation alongside manual work first? Rubric: 3 = compare results; catch errors before relying; 2 = partial; 1 = unnecessary; 0 = none.

Synthesis

Automate repetitive rules, keep humans on judgment and high impact, log everything and never automate unsafe contact or sharing.

Continue studying — chapter 22 of 30

← Chapter 21 (draft) · Course chapter index · Chapter 23 (draft) →