Chapter 19 — Threat intelligence and OSINT: monitoring
Lesson map
Lesson 19 published. Map, example, EX055.
flowchart TD
N0["Learning outcomes"]
N1["19.1 Continuous matching"]
N2["Lab — expiry review"]
N3["19.2 Reading the result"]
N4["19.4 Review, not delete"]
N5["19.5 Monitoring health"]
N6["19.6 Lookalike watch"]
N7["19.7 Thresholds"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX055"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["19.1 Continuous matching"]
N2["Lab — expiry review"]
N3["19.2 Reading the result"]
N4["19.4 Review, not delete"]
N5["19.5 Monitoring health"]
N6["19.6 Lookalike watch"]
N7["19.7 Thresholds"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX055"]
[Learning outcomes]
|
v
[19.1 Continuous matching]
|
v
[Lab — expiry review]
|
v
[19.2 Reading the result]
|
v
[19.4 Review, not delete]
|
v
[19.5 Monitoring health]
|
v
[19.6 Lookalike watch]
|
v
[19.7 Thresholds]
|
v
[Worked example] --> [Exercise EX055]
Learning outcomes
You will (1) monitor indicator matches over time; (2) apply type-specific expiry rules; (3) watch for lookalike registrations continuously; (4) report monitoring health.
19.1 Continuous matching
Matching runs daily against new logs, not just once. A domain from last month can appear today when a staff member opens an old message.
A fraud text can sit in a resident's or staff member's phone for weeks, and a phishing email can be opened long after it arrived. Matching only on the day an indicator is received would miss those later contacts. Daily matching against new logs means an indicator stays useful for as long as it is active in the store.
Lab — expiry review
"""Lab (ch 19): flag indicators for review by type-specific age limits (synthetic)."""
from datetime import date
LIMIT = {"ipv4": 30, "domain": 90, "sha256": 365}
today = date(2026, 10, 1)
items = [("198.51.100.23", "ipv4", date(2026, 8, 20)), ("pay-portal.example", "domain", date(2026, 8, 20)),
("update-svc.example", "domain", date(2026, 6, 10)), ("ab12" * 16, "sha256", date(2026, 1, 5))]
for v, t, last in items:
age = (today - last).days
print(v[:20], t, age, "REVIEW" if age > LIMIT[t] else "keep")
Run with python3 lab_expiry.py. Verified output:
198.51.100.23 ipv4 42 REVIEW
pay-portal.example domain 42 keep
update-svc.example domain 113 REVIEW
ab12ab12ab12ab12ab12 sha256 269 keep
19.2 Reading the result
With a reference date of 1 October 2026, the address last seen 42 days ago exceeds its 30-day limit and is flagged for review. The fraud domain at 42 days is within its 90-day limit and kept. update-svc.example at 113 days exceeds 90 and is flagged. The hash at 269 days is within its 365-day limit.
Lena checks the ages by hand: from 20 August to 1 October is 42 days, from 10 June is 113 and from 5 January is 269. The same age of 42 days gives different decisions for the address and the domain, which is the point of type-specific limits. The hash is printed shortened to its first 20 characters, as the lab code does with v[:20].
19.3 Why limits differ
Addresses are reassigned quickly, especially in cloud services, so stale address blocks hit innocent parties. Domains last longer; hashes identify exact files indefinitely, though the file may stop circulating. Bianco's Pyramid of Pain explains this ordering.
Bianco places hashes and addresses near the bottom of the pyramid because attackers change them cheaply, but the limits here follow a different property: how long a value keeps pointing at the same thing. An address may serve a different customer next month, while a hash always names the same file, so the hash limit can be the longest.
19.4 Review, not delete
Flagged items go to review: check fresh sightings and source updates. Expired items are marked inactive with a date, keeping history.
Review checks whether the indicator has been seen recently in MCX items or CMS logs and whether its source has updated it. If neither, the indicator is marked inactive and removed from exports, but its record stays. That way, if update-svc.example appears again in a year, Lena can see its earlier history and the vendor's original report. The inactive date is shown in the store, so nobody mistakes an expired indicator for a current one during a search.
19.5 Monitoring health
Lena reports weekly: logs received from each source, matches, flagged expiries and any source that stopped sending. A silent log source is itself an alert.
Each item in the report answers a different question: whether data is arriving, whether anything matched, what needs review and which sources have gone quiet. A source that stops sending looks the same as a clean week unless someone checks the volume, which is how the library resolver outage in the worked example was noticed.
19.6 Lookalike watch
A daily certificate-transparency query for CMS-like names runs automatically and lists new matches for triage.
The query looks for names containing corvale together with payment words, as described in chapter 16, and its results go to triage like any other item. Running it daily matters because a certificate can be issued only a day or two before the fraud texts begin, and an early match gives time to request a block and a takedown. Matches that turn out to be legitimate, such as a council supplier's new name, are recorded so the same name is not triaged again.
19.7 Thresholds
Limits are set with owners and written down. Changing them is a recorded decision, not a script edit.
The limits in the lab, 30 days for addresses, 90 for domains and 365 for hashes, are examples of such agreed values. If someone edits the script to change one, the decision has no owner and no reason on record. Writing them down with owners means a later reviewer can see who agreed them and why. When a limit changes, the date and reason are added to the lab's notes and the store, so old and new results can be compared.
Alert fatigue
If monitoring produces dozens of low-value matches daily, people stop reading. Tune by removing stale indicators and grouping repeat matches per host, so each alert represents something worth a look.
Worked example
The weekly health report shows the library resolver sent no logs for three days. The network team finds a full disk on the log forwarder. Matching for those days is rerun after repair, and the gap is noted in the report.
Synthetic case. The outage is invented.
Errors and limits
Fixed limits are crude; a still-active campaign may justify keeping an old indicator.
Exercises
- EX055 — Lab expiry. Which items did the lab flag for review, and why? Rubric: 3 = address 42>30; update-svc 113>90; 2 = one; 1 = wrong; 0 = none.
- EX056 — Address ageing. Why do addresses get the shortest limit? Rubric: 3 = reassigned quickly, esp. cloud; innocent parties; 2 = partial; 1 = random; 0 = none.
- EX057 — Silent source. What did the weekly health report catch, and what followed? Rubric: 3 = library resolver silent 3 days; disk full fixed; rerun; gap noted; 2 = partial; 1 = vague; 0 = none.
Synthesis
Monitor continuously, expire by type with review, watch for lookalikes and report the health of the monitoring itself.
Continue studying — chapter 19 of 30
← Chapter 18 (draft) · Course chapter index · Chapter 20 (draft) →