Chapter 6 — Calculus II
Lesson map
Lesson 06 published. Map, example, EX016.
flowchart TD
N0["Learning outcomes"]
N1["6.1 Why decay"]
N2["6.2 The model"]
N3["Lab 2 — decay"]
N4["6.3 Choosing half-lives"]
N5["6.4 Re-observation"]
N6["6.5 When the model misleads"]
N7["Practice note — Re-fitting half-lives"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX016"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["6.1 Why decay"]
N2["6.2 The model"]
N3["Lab 2 — decay"]
N4["6.3 Choosing half-lives"]
N5["6.4 Re-observation"]
N6["6.5 When the model misleads"]
N7["Practice note — Re-fitting half-lives"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX016"]
[Learning outcomes]
|
v
[6.1 Why decay]
|
v
[6.2 The model]
|
v
[Lab 2 — decay]
|
v
[6.3 Choosing half-lives]
|
v
[6.4 Re-observation]
|
v
[6.5 When the model misleads]
|
v
[Practice note — Re-fitting half-lives]
|
v
[Worked example] --> [Exercise EX016]
Learning outcomes
You will (1) model indicator confidence decay with an exponential function; (2) solve for the time a value crosses a threshold; (3) choose half-lives per indicator type; (4) recognise when decay models mislead.
6.1 Why decay
Indicators lose value: adversaries rotate IP addresses quickly and domains more slowly, while behaviours described by ATT&CK techniques persist for months or years. Bianco's Pyramid of Pain captures this ordering. Lantern encodes it as a confidence score that decays over time unless re-observed.
The score is not a measure of how dangerous an indicator is. It estimates how likely the indicator still points at hostile activity. An address that was malicious last week may now belong to an unrelated customer of the same hosting firm, and blocking it would cut off legitimate traffic. Decay builds that risk into Lantern without anyone having to remember to remove old entries by hand.
6.2 The model
c(t) = c₀ · e^(−t·ln2/h), where h is the half-life in days. After one half-life confidence halves; after two it quarters. Solving c(t) = floor gives t = h · ln(c₀/floor) / ln 2.
In words, the score falls by the same fraction every day, so the drop is large while confidence is high and small near the floor. The second formula answers the practical question: how many days until this indicator needs review? Kofi uses it to schedule re-validation rather than letting entries linger. Changing the half-life changes the slope, but the shape stays the same.
Lab 2 — decay
"""Lab 2 (ch 06): exponential confidence decay for a synthetic indicator, c(t) = c0 * exp(-t * ln2 / h)."""
import math
c0, half_life_days, floor = 90.0, 14.0, 30.0
for t in (0, 7, 14, 21, 28):
print(f"day {t:2d}: confidence {c0 * math.exp(-t * math.log(2) / half_life_days):5.1f}")
t_floor = half_life_days * math.log(c0 / floor) / math.log(2)
print(f"falls below {floor:.0f} after {t_floor:.1f} days -> retire or re-validate")
Run with python3 lab_decay.py. Verified output:
day 0: confidence 90.0
day 7: confidence 63.6
day 14: confidence 45.0
day 21: confidence 31.8
day 28: confidence 22.5
falls below 30 after 22.2 days -> retire or re-validate
6.3 Choosing half-lives
Half-lives are engineering parameters fitted from re-observation data: how often does an IP indicator still appear malicious after a week? Lantern starts with 14 days for IPs, 45 for domains and none for file hashes (which do not change but may become irrelevant), then adjusts from evidence.
Kofi records each starting value with the reasoning behind it. The fourteen days for addresses reflects how quickly hosting customers change; the longer period for domains reflects registration costs. Hashes do not decay because a file never changes, yet they are reviewed when the malware family stops appearing.
6.4 Re-observation
A sighting resets c to a new value. The model therefore describes staleness, not truth.
When a partner reports the same address again, or BEN's own sensors see it in hostile traffic, Lantern raises the score to the value set for that kind of sighting and the clock starts again. An indicator that keeps being seen stays active for as long as it is seen. One that is never seen again fades out on schedule, whatever its initial confidence. The record keeps every sighting with its source, so an analyst can see why the score is what it is.
6.5 When the model misleads
A dormant infrastructure reused after months will have decayed to near zero. Retired indicators should be archived, not deleted, so analysts can still match historical events.
Kofi saw this with a domain that had been quiet for six months and then reappeared in a phishing run. Its score had fallen far below the blocking floor, so Lantern did not block it automatically. Because the archive still held it, the analyst who saw the new sighting found the earlier campaign at once and raised the score by hand. Deleting retired entries would have lost that history.
Practice note — Re-fitting half-lives
Each quarter Kofi samples retired indicators and checks how many reappeared in later sightings. If IP indicators are still being sighted after 30 days at a meaningful rate, the 14-day half-life is too short. The refit is recorded with its sample size so later reviewers can judge whether the change was justified.
Worked example
With c₀ = 90, h = 14 days and a floor of 30, the lab shows confidence 45.0 at day 14 and crossing the floor after 22.2 days. Lantern schedules re-validation on day 22 and removes the indicator from automated blocking unless re-observed.
Synthetic case. Parameters are illustrative, not measured.
Errors and limits
A neat curve can hide poor data; fit half-lives from evidence. Decay suits indicators, not judgments about adversary intent.
Exercises
- EX016 — Day-14 value. What confidence does the lab print at day 14, and why that number? Rubric: 3 = 45.0; one half-life halves 90; 2 = number only; 1 = wrong; 0 = none.
- EX017 — Floor time. Using the formula, after how many days does confidence cross 30? Rubric: 3 = 22.2 days with formula; 2 = number only; 1 = 28; 0 = none.
- EX018 — Archive not delete. Why archive decayed indicators rather than deleting them? Rubric: 3 = historical matching/reuse of infrastructure; retrospective analysis; 2 = vague; 1 = delete is fine; 0 = none.
Synthesis
Exponential decay encodes staleness, half-lives come from evidence, and retired indicators stay searchable.
Continue studying — chapter 06 of 50
← Chapter 05 (draft) · Course chapter index · Chapter 07 (draft) →