◇ OpenCyber Academy
Engineering · Chapter 31 · English

Chapter 31 — Threat intelligence and OSINT: implementation II

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 31 — Threat intelligence and OSINT: implementation II

Lesson map

Lesson 31 published. Map, example, EX091.

flowchart TD
  N0["Learning outcomes"]
  N1["31.1 Enrichment"]
  N2["31.2 Sightings"]
  N3["31.3 Confidence on read"]
  N4["31.4 Publishing"]
  N5["31.5 Personal data in telemetry"]
  N6["Practice note — rollout"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX091"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["31.1 Enrichment"]
  N2["31.2 Sightings"]
  N3["31.3 Confidence on read"]
  N4["31.4 Publishing"]
  N5["31.5 Personal data in telemetry"]
  N6["Practice note — rollout"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX091"]

[Learning outcomes]
    |
    v
[31.1 Enrichment]
    |
    v
[31.2 Sightings]
    |
    v
[31.3 Confidence on read]
    |
    v
[31.4 Publishing]
    |
    v
[31.5 Personal data in telemetry]
    |
    v
[Practice note — rollout]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX091]

Learning outcomes

You will (1) implement enrichment with caching and provider-terms compliance; (2) implement sighting ingestion from internal telemetry; (3) compute decayed confidence on read; (4) publish analyst-approved objects back to partners.

31.1 Enrichment

Enrichment adds context: internal asset ownership for an address, passive DNS history for a domain, ATT&CK technique links for a malware family. External enrichment providers have terms on query volume, retention and redistribution. Lantern's enricher records which provider supplied each field so that redistribution rules can be enforced at export.

Internal enrichment is often the most useful kind. Knowing that a contacted address belongs to a substation engineer's laptop changes the priority of a sighting at once. Kofi's enricher queries internal sources first.

31.2 Sightings

When BEN's DNS resolver or proxy sees an indicator, Lantern records a STIX 2.1 sighting object referencing the indicator, with first and last seen times and a count. Sightings raise confidence and give partners valuable feedback when shared under agreement.

Kofi's sighting writer groups repeated hits within an hour into one object with a count, so a busy indicator does not produce thousands of records. The object records which BEN sensor saw it, using an internal identifier rather than a hostname. That keeps the internal network layout out of anything that might later be shared.

31.3 Confidence on read

Rather than rewriting every record nightly, Lantern stores c₀ and the last-sighting time and computes decayed confidence when queried. This avoids mass updates and keeps the model change in one function.

Computing on read also means that changing a half-life takes effect immediately for every indicator, with no migration. The cost is a little processing at query time, which Kofi measured as negligible next to the lookup itself. Exports and the firewall push call the same function, so every consumer sees the same score for the same indicator at the same moment. A test compares the function's results with the decay lab's printed values.

31.4 Publishing

Analysts approve objects for sharing with a TLP marking. The publisher strips enrichment fields whose provider forbids redistribution, signs the bundle and adds it to the community TAXII collection.

The approval step is where an analyst checks that the object says what it should and that its label is right. The publisher then applies the mechanical rules without exceptions. Kofi's tests include an object with a licensed passive DNS field, and they fail if that field appears in the signed bundle. Analysts can see in the interface which fields will be removed before they approve.

31.5 Personal data in telemetry

Sightings come from systems that know which employee's machine made a request. Lantern stores the asset identifier, not the person, and shares only aggregate counts outward.

An analyst investigating a sighting can still find the person through BEN's asset register, but that lookup happens in the SOC's own tools under their access rules, not in Lantern. Kofi made the design choice after the privacy officer pointed out that Lantern's data is shared more widely than SOC case files. Outbound reports give counts per asset type, such as workstations or servers, and never a list of machines.

Practice note — rollout

New stages roll out behind a feature flag to a fraction of traffic first. Kofi compares metrics between flagged and unflagged paths for a week before full rollout.

Worked example

A partner asks for sightings of their indicators. Lantern exports sighting objects with counts and time ranges but without internal hostnames or user names, marked TLP:AMBER per the partner agreement. The partner reports that two of their indicators had no sightings across the community and retires them.

Synthetic case. The exchange is invented.

Errors and limits

Redistributing licensed enrichment can breach contracts. Sightings can reveal a member's exposure; share only as agreed.

Exercises

Synthesis

Enrich within provider terms, record sightings without people, compute decay on read, and publish only what may be shared.

Continue studying — chapter 31 of 50

← Chapter 30 (draft) · Course chapter index · Chapter 32 (draft) →