Chapter 34 — Comparative project | Bachelor's Threat intelligence and OSINT

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 34 published. Map, example, EX100.

flowchart TD
  N0["Learning outcomes"]
  N1["34.1 The project"]
  N2["34.2 Fair criteria"]
  N3["34.3 Evidence"]
  N4["34.4 Scoring"]
  N5["34.5 Recommendation"]
  N6["34.6 Comparing like with like"]
  N7["Stakeholder input"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX100"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["34.1 The project"]
  N2["34.2 Fair criteria"]
  N3["34.3 Evidence"]
  N4["34.4 Scoring"]
  N5["34.5 Recommendation"]
  N6["34.6 Comparing like with like"]
  N7["Stakeholder input"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX100"]

[Learning outcomes]
    |
    v
[34.1 The project]
    |
    v
[34.2 Fair criteria]
    |
    v
[34.3 Evidence]
    |
    v
[34.4 Scoring]
    |
    v
[34.5 Recommendation]
    |
    v
[34.6 Comparing like with like]
    |
    v
[Stakeholder input]
    |
    v
[Worked example] --> [Exercise EX100]

Learning outcomes

You will (1) compare two intelligence-sharing arrangements; (2) build fair comparison criteria; (3) gather evidence lawfully; (4) recommend with stated trade-offs.

34.1 The project

Students compare two arrangements: a sector sharing community like HSSC and a commercial intelligence feed. Criteria include relevance to health, timeliness, format (STIX/TAXII support), handling rules, cost, licence terms and feedback mechanisms.

The comparison uses HCHT's own needs as the frame, since a feed that suits a bank may not suit a health trust. Students describe each arrangement in the same structure before scoring, so the reader understands both before seeing any numbers. Amara's group agrees the criteria with the lead first.

34.2 Fair criteria

Define criteria before looking at evidence, so you do not pick criteria that favour a preferred answer. Weight them by HCHT's requirements, and publish the weights.

Amara's group sets weights with the security lead, giving the most weight to health relevance and handling rules. The weights are published in the report with a sentence explaining each. A reviewer who disagrees can change a weight and see whether the recommendation changes, which makes the debate about priorities rather than about hidden scoring choices.

34.3 Evidence

Use documentation, public descriptions, membership agreements available to HCHT and interviews with consenting staff. Do not request trial accounts under false pretences or share HCHT data to test a vendor.

Amara's group uses the commercial vendor's public documentation and HCHT's existing contract terms, and interviews two HCHT analysts who have used both sources. They do not ask the vendor for a trial, since that could create obligations HCHT has not approved. Evidence is recorded in a table with its source for each criterion, so gaps are visible.

34.4 Scoring

Score each criterion 0–3 with written evidence. Sensitivity check: does the recommendation change if one weight changes? If yes, say so.

Amara's group scores each criterion with one sentence of evidence and runs the sensitivity check by doubling the weight on cost. The recommendation holds, which the report states. A criterion with no evidence is scored as unknown rather than guessed, and the report notes how it could be checked. The scoring table is attached to the report in full, with every evidence sentence, so readers can check any score.

34.5 Recommendation

Recommend with trade-offs: "Keep HSSC as primary for health relevance; add the commercial feed only for vulnerability exploitation data if budget allows."

The recommendation states what would change it, such as a drop in HSSC's timeliness or a cheaper feed. Amara's group presents it to the security lead with the full scoring table attached. The lead asks for the vulnerability data to be trialled through a free public source first, and the report is updated to record that decision. The lead reads it within a week.

34.6 Comparing like with like

The community shares analysed reports; the feed delivers bulk indicators. Comparing raw volume favours the feed unfairly. Normalise where you can — for example, items that matched HCHT's inventory per month — so the comparison reflects usefulness, not size.

Amara's group uses this normalised measure. For the commercial feed, she counts indicators that matched HCHT's products or logs in a month; for HSSC, she counts reports that led to an action. Both are small numbers, but they are comparable, and the feed's apparent advantage disappears. The report explains the measure and its limits.

Stakeholder input

Ask the network team, service desk and supervisor which criteria matter to them and why. Their answers shape weights and reveal hidden requirements — for example, the network team may need indicators in a format their firewall imports directly. Record each stakeholder's view so the final weights can be traced back to real needs rather than the analyst's taste.

Worked example

Amara's scoring gives HSSC 2.4 and the commercial feed 2.1. When she raises timeliness weight, the feed edges ahead. Her report states that the choice depends on how much HCHT values speed over sector relevance, and suggests a three-month trial measurement.

Synthetic case. The scores are invented.

Errors and limits

Vendor claims are marketing until verified. Small trials may not reflect long-term value.

Exercises

EX100 — Criteria timing. Why define criteria before gathering evidence?
Show marking criteria
Rubric: 3 = avoids picking criteria to favour a preferred answer; 2 = partial; 1 = order irrelevant; 0 = none.
EX101 — Sensitivity. What did the sensitivity check reveal, and how did Amara report it?
Show marking criteria
Rubric: 3 = timeliness weight flips result; stated dependency and proposed trial; 2 = one; 1 = vague; 0 = none.
EX102 — Unfair metric. Why is raw indicator volume an unfair comparison?
Show marking criteria
Rubric: 3 = favours bulk feed; volume ≠ usefulness; normalise by relevance; 2 = partial; 1 = fair; 0 = none.

Synthesis

Fix criteria first, gather evidence lawfully, score with reasons, test sensitivity and recommend with trade-offs.

Continue studying — chapter 34 of 40

← Chapter 33 (draft) · Course chapter index · Chapter 35 (draft) →