Chapter 7 — Threat intelligence and OSINT: data and telemetry
Lesson map
Lesson 07 published. Map, example, EX025.
flowchart TD
N0["Learning outcomes"]
N1["7.1 Telemetry list"]
N2["7.2 Coverage"]
N3["7.3 Retention"]
N4["7.4 Access"]
N5["7.5 Minimal extracts"]
N6["7.6 Clock accuracy"]
N7["7.7 Quality checks"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX025"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["7.1 Telemetry list"]
N2["7.2 Coverage"]
N3["7.3 Retention"]
N4["7.4 Access"]
N5["7.5 Minimal extracts"]
N6["7.6 Clock accuracy"]
N7["7.7 Quality checks"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX025"]
[Learning outcomes]
|
v
[7.1 Telemetry list]
|
v
[7.2 Coverage]
|
v
[7.3 Retention]
|
v
[7.4 Access]
|
v
[7.5 Minimal extracts]
|
v
[7.6 Clock accuracy]
|
v
[7.7 Quality checks]
|
v
[Worked example] --> [Exercise EX025]
Learning outcomes
You will (1) list telemetry sources that confirm or refute intelligence; (2) judge each source's coverage; (3) request log access properly; (4) keep extracts minimal.
7.1 Telemetry list
DNS resolver logs, proxy logs, email gateway logs, firewall logs, directory sign-ins, endpoint process events on staff machines, learning-platform login logs. Each answers different questions.
Tomás keeps the list as a table with the questions each source can answer. DNS shows which devices asked for a domain; the proxy shows staff web visits; email logs show delivery; sign-ins show account use. Endpoint events show what ran on staff machines, and learning-platform logs show logins and administrator changes. Tomás adds the retention period and the owning team to each row of the table.
7.2 Coverage
DNS covers all campus devices; endpoint events cover only college-managed machines; the proxy covers staff only. Tomás writes coverage beside each source so negative results are honest.
A clean DNS search means no campus device asked for the domain within the retention period, which is a strong statement. A clean proxy search says nothing about students, because they never pass through it. Writing coverage beside each result stops readers treating a narrow negative as proof that the whole college is safe.
7.3 Retention
DNS 90 days, proxy 30 days, cloud sign-ins 30 days, endpoint 60 days. An indicator from 45 days ago can still be checked in DNS but not in the proxy.
Tomás checks an indicator's age against these periods before searching, so he knows which logs can still answer. An ESX report about activity 40 days ago can be tested in DNS and endpoint logs, but not in the proxy or cloud sign-ins. He states which logs were too short, rather than reporting a plain negative.
7.4 Access
Tomás has read-only access to DNS and email logs. Others are searched on request by owning teams. Access requests are justified in writing.
Read-only access lets Tomás match indicators without any risk of changing records. For other sources, a written request names the indicator, the time range and the fields needed, and the owning team runs the search. Their answer goes into the ticket, so the evidence trail shows who searched what and when. Each written request also states how long the requested extract will be kept and who will read it.
7.5 Minimal extracts
When evidence is needed, export only matching lines with needed fields — time, host, indicator — not full logs with every user's browsing.
In the DNS case from the library, the extract held two lines with time, device and domain, which was enough to show the lookups. Full logs would have included every other lookup by library users that morning. Keeping extracts small reduces the personal data stored in tickets and makes the evidence easier to review. The ticket records which fields were exported, so nobody has to guess what was left out of the extract.
7.6 Clock accuracy
All servers synchronise time. A server with drifting time makes correlation unreliable; Tomás reports such drift.
Correlation depends on comparing times across sources, such as DNS lookups and firewall connections. A resolver that drifted by a few minutes could make a lookup appear after the connection it caused. Tomás reports drift to the owning team and notes it beside any affected result, so readers allow for it. Endpoint agents on staff machines take their time from the same servers, which keeps endpoint process events comparable with DNS and proxy records.
7.7 Quality checks
Daily volume per source shows gaps: a resolver that logged nothing overnight probably failed.
Tomás compares each day's line count with the usual range for that source and weekday. A sudden drop to zero is usually a logging fault, not a quiet night, and it means searches covering that period are incomplete. He reports the gap to the owner and records it, so later negative results are qualified. Weekend and holiday volumes are lower, so the usual ranges are kept separately for term-time weekdays, weekends and college holidays.
7.8 Telemetry for techniques
Technique intelligence needs behavioural data. Phishing needs email logs; valid-account abuse needs sign-ins; command channels need DNS and proxy. A matrix of techniques against sources shows blind spots.
The matrix lists techniques from ATT&CK down one side and Ashgrove's log sources across the top. A technique with no source marked is a blind spot, such as scripts running on student laptops. Tomás shares the matrix with his supervisor, who decides which blind spots to accept and which to close.
Learning-platform logs
The learning platform records logins, file downloads and administrator changes. Intelligence about attackers abusing course platforms to host malicious files makes upload logs relevant too; Tomás asks the e-learning team which fields they keep and for how long.
Worked example
ESX reports a domain used in attacks 40 days ago. Tomás searches DNS (90-day retention): two lookups from the library. The proxy has no record because its retention is 30 days. His note states both facts so readers do not misread the proxy's silence.
Synthetic case. The domain and lookups are invented.
Errors and limits
Telemetry shows activity, not intent; interpret with care.
Exercises
- EX025 — Retention check. Why was the proxy silent about the 40-day-old domain? Rubric: 3 = 30-day retention; 2 = partial; 1 = no visits; 0 = none.
- EX026 — Minimal extract. What should an evidence extract contain? Rubric: 3 = only matching lines with needed fields; 2 = partial; 1 = full logs; 0 = none.
- EX027 — Source coverage. Which source covers all campus devices, and which only managed machines? Rubric: 3 = DNS all; endpoint managed only; 2 = one; 1 = wrong; 0 = none.
- EX028 — Technique matrix. What does a technique-by-source matrix reveal? Rubric: 3 = blind spots where no telemetry covers a technique; 2 = partial; 1 = vague; 0 = none.
Synthesis
Know each source's coverage and retention, request access properly, extract minimally and map techniques to telemetry.
Continue studying — chapter 07 of 18
← Chapter 06 (draft) · Course chapter index · Chapter 08 (draft) →