Chapter 11 — Applied project | Postgraduate Threat intelligence and OSINT

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 11 published. Map, example, EX051.

flowchart TD
  N0["Learning outcomes"]
  N1["11.1 Choosing the project"]
  N2["11.2 Scope and ethics"]
  N3["11.3 Method"]
  N4["11.4 Evaluation"]
  N5["11.5 Stakeholders"]
  N6["11.6 Reporting"]
  N7["Safe detection testing"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX051"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["11.1 Choosing the project"]
  N2["11.2 Scope and ethics"]
  N3["11.3 Method"]
  N4["11.4 Evaluation"]
  N5["11.5 Stakeholders"]
  N6["11.6 Reporting"]
  N7["Safe detection testing"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX051"]

[Learning outcomes]
    |
    v
[11.1 Choosing the project]
    |
    v
[11.2 Scope and ethics]
    |
    v
[11.3 Method]
    |
    v
[11.4 Evaluation]
    |
    v
[11.5 Stakeholders]
    |
    v
[11.6 Reporting]
    |
    v
[Safe detection testing]
    |
    v
[Worked example] --> [Exercise EX051]

Learning outcomes

You will (1) deliver a scoped CTI improvement project at work; (2) apply evaluation methods from earlier chapters; (3) manage stakeholders and ethics; (4) report outcomes honestly.

11.1 Choosing the project

Rafael's project: build an intelligence-driven detection backlog. Using ATT&CK-coded FPSC reports and NLG incidents, rank techniques by sector frequency, NLG exposure and current detection coverage, then deliver the top ten as detection requirements to the operations centre. The project answers a decision the operations centre faces each quarter: which detections to build next. Ranking by sector frequency, NLG exposure and current coverage combines intelligence with knowledge of NLG's own systems. Delivering ten requirements rather than a long list keeps the backlog realistic for the detection team.

11.2 Scope and ethics

In scope: public and FPSC reports (respecting TLP), internal incident summaries, current detection inventory. Out of scope: offensive testing, adversary infrastructure interaction, personal data beyond what incidents already hold. Respecting TLP means FPSC content marked AMBER or AMBER+STRICT stays out of anything shared beyond permitted recipients, including the academic report. Excluding offensive testing and adversary infrastructure keeps the project within NLG's collection policy. Personal data is limited to what incident records already hold, as the privacy officer agreed.

The privacy officer signs off the scope before coding begins.

11.3 Method

Code reports to techniques with a second coder on a sample; compute frequencies; score exposure with asset owners; mark coverage as none, partial or full with the detection team. Combine scores transparently. A second coder on a sample checks that technique coding is consistent, and disagreements are discussed and recorded. Asset owners score exposure because they know which systems face the internet or suppliers. The detection team marks coverage because it knows what rules exist. Rafael publishes the scoring formula with the backlog, so anyone can see why an item ranked where it did.

11.4 Evaluation

Success measures: number of backlog items implemented within the term; detection coverage change; time from intelligence to detection requirement. Compare with the previous quarter. Each measure tests a different part of the project's value: implementation shows whether the operations centre used the backlog, coverage change shows whether NLG can see more techniques, and time to requirement shows whether intelligence reaches detection faster. Comparing with the previous quarter gives a baseline, and Rafael reports counts rather than percentages alone.

Rafael set these measures with the operations lead before work began.

11.5 Stakeholders

Agree with the operations lead how many requirements the team can absorb. Ten realistic items beat thirty ignored ones. The operations lead knows how many detections the team can build in a term. Agreeing the number in advance means the backlog is sized to capacity, and items that are not delivered are visible as carried over rather than quietly dropped. Rafael and the lead review progress fortnightly, and seven of the ten items shipped within the term.

The operations lead also agrees which detections need testing time in the shared calendar.

11.6 Reporting

The final report states what was delivered, measured effects, limits and next steps. Restricted FPSC content stays out of the academic version. Rafael's report states that seven of ten items shipped, describes the coverage change, and lists limits such as the small sample of coded reports. Next steps include the remaining three items, which carry over to the next quarter's backlog. The academic version describes FPSC reporting only in general terms, so no restricted content leaves NLG and FPSC.

His supervisors review both the academic and the executive versions before submission.

Safe detection testing

Testing new detections should use approved, harmless simulations — for example, an administrator performing a scripted unusual login from a test account — scheduled with the operations team and recorded in advance so nobody mistakes the test for a real incident.

Worked example

Coding 40 reports yields Valid Accounts (T1078) and Remote Services (T1021) as high-frequency techniques with partial coverage at NLG. Exploit Public-Facing Application (T1190) has low frequency but high exposure because of the customs portal. The backlog places T1078 anomalies, supplier remote-service monitoring and portal exploitation alerts in the top three. Seven of ten items ship within the term.

Synthetic case. The counts and outcomes are invented.

Errors and limits

Coverage self-assessments are optimistic; test detections with safe simulations approved by the operations team.

Exercises

EX051 — Ranking factors. Which three factors ranked the backlog?
Show marking criteria
Rubric: 3 = sector frequency, NLG exposure, detection coverage; 2 = two; 1 = one; 0 = none.
EX052 — Low-frequency item. Why did T1190 rank high despite low frequency?
Show marking criteria
Rubric: 3 = customs portal exposure; 2 = partial; 1 = random; 0 = none.
EX053 — Absorbable backlog. Why agree capacity with the operations lead?
Show marking criteria
Rubric: 3 = realistic delivery; ignored items waste effort; 2 = partial; 1 = unnecessary; 0 = none.
EX054 — Success measures. Give two measurable success criteria for the project.
Show marking criteria
Rubric: 3 = items implemented, coverage change, time to requirement; 2 = one; 1 = vague; 0 = none.
EX055 — Academic version. What must be removed from the academic report?
Show marking criteria
Rubric: 3 = restricted FPSC/TLP content; sensitive NLG detail; 2 = partial; 1 = vague; 0 = none.

Synthesis

Scope a useful project, use transparent scoring, agree capacity, measure outcomes and keep restricted material out of academic outputs.

Continue studying — chapter 11 of 12

← Chapter 10 (draft) · Course chapter index · Chapter 12 (draft) →