◇ OpenCyber Academy
Postgraduate · Chapter 3 · English

Chapter 3 — Threat intelligence and OSINT: specialist evidence

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 3 — Threat intelligence and OSINT: specialist evidence

Lesson map

Lesson 03 published. Map, example, EX011.

flowchart TD
  N0["Learning outcomes"]
  N1["3.1 Why ACH"]
  N2["Lab — competing hypotheses"]
  N3["3.2 Reading the result"]
  N4["3.3 Diagnosticity"]
  N5["3.4 Deception and gaps"]
  N6["3.5 Confidence"]
  N7["3.6 Sensitivity"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX011"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["3.1 Why ACH"]
  N2["Lab — competing hypotheses"]
  N3["3.2 Reading the result"]
  N4["3.3 Diagnosticity"]
  N5["3.4 Deception and gaps"]
  N6["3.5 Confidence"]
  N7["3.6 Sensitivity"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX011"]

[Learning outcomes]
    |
    v
[3.1 Why ACH]
    |
    v
[Lab — competing hypotheses]
    |
    v
[3.2 Reading the result]
    |
    v
[3.3 Diagnosticity]
    |
    v
[3.4 Deception and gaps]
    |
    v
[3.5 Confidence]
    |
    v
[3.6 Sensitivity]
    |
    v
[Worked example] --> [Exercise EX011]

Learning outcomes

You will (1) apply analysis of competing hypotheses (ACH); (2) weigh diagnostic versus non-diagnostic evidence; (3) handle deception and gaps; (4) state conclusions with calibrated confidence.

3.1 Why ACH

Heuer's analysis of competing hypotheses asks analysts to list all reasonable hypotheses, score each piece of evidence against each, and favour the hypothesis with the least inconsistent evidence. It counters the habit of accepting the first plausible story. At NLG, the first plausible story after the data theft was an external extortion crew, because extortion emails followed. ACH forced Rafael to test that story alongside supplier compromise and insider misuse, using the same evidence for each. The matrix is kept with the assessment, so reviewers can see how each item was scored.

Lab — competing hypotheses

"""Lab (ch 03): analysis of competing hypotheses - count inconsistencies per hypothesis (synthetic)."""
H = ["H1 criminal extortion crew", "H2 insider misuse", "H3 supplier compromise"]
# C = consistent, I = inconsistent, N = neutral
E = {"E1 login from supplier VPN range": "NIC",
     "E2 data staged at 02:00 local": "CCC",
     "E3 extortion note received": "CNN",
     "E4 credentials used were service account": "ICC",
     "E5 same tool seen at two other firms": "CIC"}
score = [0, 0, 0]
for e, row in E.items():
    for k, v in enumerate(row):
        score[k] += v == "I"
for h, s in sorted(zip(H, score), key=lambda x: x[1]):
    print(f"{h}: {s} inconsistent")
print("least inconsistent:", min(zip(score, H))[1])

Run with python3 lab_ach.py. Verified output:

H3 supplier compromise: 0 inconsistent
H1 criminal extortion crew: 1 inconsistent
H2 insider misuse: 2 inconsistent
least inconsistent: H3 supplier compromise

3.2 Reading the result

Three hypotheses about a data theft at NLG are tested against five evidence items. Supplier compromise (H3) has no inconsistent evidence; criminal extortion (H1) has one; insider misuse (H2) has two. H3 is the leading hypothesis, but H1 is close. Close scores mean the ranking could change with one new piece of evidence, so Rafael treats H3 as the leading explanation, not a conclusion. His report keeps H1 as plausible and names verification of the supplier VPN log as the most valuable next step.

3.3 Diagnosticity

Evidence consistent with every hypothesis — data staged at 02:00 (E2) — does not help choose. Diagnostic evidence discriminates: the login from the supplier VPN range (E1) is inconsistent with an external crew and neutral for an insider. Analysts should collect more diagnostic evidence rather than more evidence. For Rafael, this means the next effort should go to verifying the supplier VPN log with the supplier, not collecting more logs that fit every hypothesis. He lists which evidence items are diagnostic and which are not in the assessment, so leadership can see why one verification step matters more than many additional logs.

3.4 Deception and gaps

Adversaries plant false clues; tools are sold widely. Ask which evidence would be easy to fake and give it less weight. List what evidence is missing and what it would show. In the NLG case, a VPN address could be borrowed by an attacker using stolen supplier credentials, so E1 does not prove the supplier itself acted. Rafael notes this beside E1 and lists missing evidence, such as the supplier's own VPN log, with what each would show. Gaps are written into the assessment itself, so leadership knows what the conclusion does not yet rest on.

3.5 Confidence

A one-point margin between H3 and H1 supports low-to-moderate confidence. ICD 203 asks analysts to explain the basis of confidence, including gaps. Rafael's report states the basis for low-to-moderate confidence: the close margin, reliance on one diagnostic item and missing supplier logs. Explaining confidence lets leadership decide how much to act on the assessment. The supplier access restrictions went ahead because they were sensible under either leading hypothesis.

He also says what would raise confidence: confirmation from the supplier's own VPN log that the session came from its staff, or evidence that the supplier credentials had been stolen elsewhere.

3.6 Sensitivity

Ask which single evidence item, if wrong, would change the ranking. If E1's VPN attribution is mistaken, H1 and H3 tie. That tells Rafael where to verify first. Sensitivity analysis turns a list of evidence into a plan for verification. Because E1 decides the ranking, Rafael asks the supplier to check its VPN log before spending time elsewhere. He records the result of the check when it arrives, and updates the matrix and the confidence statement in a marked revision of the assessment.

Worked example

Rafael's report states: "The leading explanation is a compromise of a supplier's access (low-to-moderate confidence). An external extortion crew remains plausible. Verifying the supplier VPN log (E1) with the supplier is the most valuable next step." He shares the assessment with leadership only, at TLP:AMBER+STRICT.

Synthetic case. The theft and evidence are invented.

Errors and limits

ACH counts are judgments; equal weights hide strong evidence. Use counts to structure debate, not as verdicts.

Exercises

Synthesis

Test competing hypotheses, seek diagnostic evidence, guard against deception and calibrate confidence to margins and gaps.

Continue studying — chapter 03 of 12

← Chapter 02 (draft) · Course chapter index · Chapter 04 (draft) →