Chapter 17 — Threat intelligence and OSINT: tooling
Lesson map
Lesson 17 published. Map, example, EX049.
flowchart TD
N0["Learning outcomes"]
N1["17.1 The CMS stack"]
N2["17.2 Scripts or platform"]
N3["17.3 Securing tools"]
N4["17.4 API keys"]
N5["17.5 Maintenance"]
N6["17.6 Choosing new tools"]
N7["17.7 Licences"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX049"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["17.1 The CMS stack"]
N2["17.2 Scripts or platform"]
N3["17.3 Securing tools"]
N4["17.4 API keys"]
N5["17.5 Maintenance"]
N6["17.6 Choosing new tools"]
N7["17.7 Licences"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX049"]
[Learning outcomes]
|
v
[17.1 The CMS stack]
|
v
[17.2 Scripts or platform]
|
v
[17.3 Securing tools]
|
v
[17.4 API keys]
|
v
[17.5 Maintenance]
|
v
[17.6 Choosing new tools]
|
v
[17.7 Licences]
|
v
[Worked example] --> [Exercise EX049]
Learning outcomes
You will (1) describe the tools in a small CTI stack; (2) choose between scripts and platforms; (3) secure the tools themselves; (4) maintain tools sustainably.
17.1 The CMS stack
Lena's stack: standard-library Python scripts for parsing and matching, a small open-source threat intelligence platform that stores STIX 2.1 and pulls MCX's TAXII 2.1 collection, the log search tool owned by operations, and a shared ticket queue.
Each part has a clear job: scripts do the checks Lena can read line by line, the platform holds and shares indicators in STIX with their TLP marks, the log search tool answers matching questions and the ticket queue carries requests to owners. Because operations owns the log search tool, Lena's searches run under their rules.
17.2 Scripts or platform
Scripts are flexible and transparent but depend on their author. A platform offers storage, sharing, TLP marking and a web interface but needs maintenance. CMS uses the platform for storage and sharing and scripts for local checks.
The split between platform and scripts plays to each side's strength. The platform keeps TLP marks and partner data in one controlled place, which a folder of scripts would do badly; scripts handle quick local tasks such as refanging or deduplicating, where Lena wants to see every step. Neither has to do everything at CMS, which keeps both of them simpler.
17.3 Securing tools
The platform holds TLP:AMBER partner data, so it needs strong authentication, restricted access, updates and backups. A compromised CTI platform would leak partner trust.
A leak of TLP:AMBER items would expose what other councils shared in confidence, and MCX members might stop sharing with CMS. That loss of trust would last longer than any technical fix. Strong authentication and restricted access therefore protect partners' data as much as CMS's own. Access is limited to the security team, and each account on the platform is reviewed when staff join or leave.
17.4 API keys
Keys for MCX's TAXII server live in the platform's secret store, not in scripts or tickets. Rotating them is a scheduled task.
A key pasted into a script or ticket can be copied by anyone who sees it, and remains valid until someone notices. The secret store limits who can read the key and records its use. Scheduled rotation means a key that leaked unnoticed stops working after a known period. Rotation dates appear in the tool register, so a missed rotation is visible to whoever owns the platform that month.
17.5 Maintenance
Each tool has an owner, an update schedule and documentation. Lena writes a README for every script: purpose, inputs, outputs, example run.
A README written at the time the script is created costs minutes, while working out an undocumented script a year later can take hours. The example run lets a colleague check that the script still behaves as described. Owners and update schedules apply to the platform as well as to Lena's scripts. The documentation is kept beside the code in the repository, so a change to a script and its README are reviewed together.
17.6 Choosing new tools
Before adding a tool, ask what requirement it serves, who maintains it, what data it touches and how it will be retired. Many small teams accumulate tools nobody uses.
The retirement question is often skipped, which is how small teams end up with tools nobody uses but that still hold data. The worked example below applies the data question to a browser extension and finds the answer unacceptable. Asking all four questions before adoption costs less than removing a tool later.
17.7 Licences
Open-source licences and commercial terms both apply. Check before redistributing tool outputs or modifying code for partners.
A partner may forbid redistribution of its data, and a tool's licence may limit how its outputs can be shared. If Lena modifies an open-source parser and gives it to MCX members, the licence decides what she must include. Checking first avoids breaching terms CMS agreed to when it adopted the tool. Lena records the licence of each tool in the tool register beside its owner, so the check is quick when a question arises later.
Testing tool changes
Before upgrading the platform, the team restores last night's backup to a test machine and upgrades that first. If imports or exports break, they find out without losing partner data or interrupting the daily feed to defensive tools.
Worked example
A colleague finds a browser extension that "auto-checks indicators against 20 online services". Lena reviews it: it sends every highlighted value to external sites, including TLP:AMBER material. She declines it and documents why in the tool register.
Synthetic case. The extension is invented.
Errors and limits
Tools encode assumptions; read their documentation before trusting outputs.
Exercises
- EX049 — Extension risk. Why was the browser extension declined? Rubric: 3 = sends restricted data to external services; TLP breach; 2 = partial; 1 = slow; 0 = none.
- EX050 — Key storage. Where should TAXII keys be stored, and where not? Rubric: 3 = secret store; not scripts/tickets; 2 = one; 1 = in script; 0 = none.
- EX051 — Script README. List the four parts of Lena's script README. Rubric: 3 = purpose, inputs, outputs, example run; 2 = three; 1 = one; 0 = none.
Synthesis
Keep the stack small, secure and documented; choose tools by requirement and protect partner data inside them.
Continue studying — chapter 17 of 30
← Chapter 16 (draft) · Course chapter index · Chapter 18 (draft) →