Chapter 4 — Network and communications security: competing explanations
Lesson map
Lesson 04 published. Map, example, EX021.
flowchart TD
N0["Learning outcomes"]
N1["4.1 Why alternatives matter"]
N2["Lab — Normalising by traffic"]
N3["4.2 Reading the normalised series"]
N4["4.3 Testing other explanations"]
N5["Analyst behaviour"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX021"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["4.1 Why alternatives matter"]
N2["Lab — Normalising by traffic"]
N3["4.2 Reading the normalised series"]
N4["4.3 Testing other explanations"]
N5["Analyst behaviour"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX021"]
[Learning outcomes]
|
v
[4.1 Why alternatives matter]
|
v
[Lab — Normalising by traffic]
|
v
[4.2 Reading the normalised series]
|
v
[4.3 Testing other explanations]
|
v
[Analyst behaviour]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX021]
Learning outcomes
You will (1) identify alternative explanations for falling alerts; (2) test them with data; (3) report honestly.
4.1 Why alternatives matter
If alerts fall after tuning, tuning is the obvious explanation but not the only one. Traffic might have dropped, a sensor might have failed, a noisy server might have been retired, or analysts might have changed how alerts are exported. A dissertation that ignores these risks claiming an effect that is not there.
A drop in alerts could come from tuning, but also from lower traffic, a holiday period or a sensor outage. Ruling these out strengthens the claim.
Each is checked in turn.
Lab — Normalising by traffic
"""Alerts per 100 GB of traffic over 8 weeks, to separate tuning from traffic changes (synthetic, stdlib only)."""
alerts = [4210, 4050, 4390, 4120, 1180, 1090, 1240, 1150]
gb = [820, 790, 860, 805, 610, 590, 640, 600]
for w, (a, g) in enumerate(zip(alerts, gb), 1):
print(f"week {w} {'pre ' if w<=4 else 'post'} alerts {a:5d} traffic {g:4d} GB per100GB {a/g*100:6.1f}")
pre = sum(a/g for a, g in zip(alerts[:4], gb[:4]))/4*100; post = sum(a/g for a, g in zip(alerts[4:], gb[4:]))/4*100
print(f"mean per100GB pre {pre:.1f} post {post:.1f}; traffic fell {1-sum(gb[4:])/sum(gb[:4]):.1%}")
Run with python3 lab_normalise.py. Verified output:
week 1 pre alerts 4210 traffic 820 GB per100GB 513.4
week 2 pre alerts 4050 traffic 790 GB per100GB 512.7
week 3 pre alerts 4390 traffic 860 GB per100GB 510.5
week 4 pre alerts 4120 traffic 805 GB per100GB 511.8
week 5 post alerts 1180 traffic 610 GB per100GB 193.4
week 6 post alerts 1090 traffic 590 GB per100GB 184.7
week 7 post alerts 1240 traffic 640 GB per100GB 193.8
week 8 post alerts 1150 traffic 600 GB per100GB 191.7
mean per100GB pre 512.1 post 190.9; traffic fell 25.5%
4.2 Reading the normalised series
Raw alerts fall from about 4,200 to 1,200 per week, a 72 percent drop. But traffic also fell by 25.5 percent in the post period, because a seasonal customer moved to another provider. Normalised alerts fall from 512 to 191 per hundred gigabytes, still a large reduction, so traffic explains only part of the raw change. The weekly series also shows a sharp step at week five, matching the tuning date, rather than a gradual decline.
Raw counts alone would credit tuning with the whole 72 percent drop; normalising separates out the volume lost with the departed customer.
Normalising per unit of traffic removes volume effects.
4.3 Testing other explanations
Sensor health logs show no outages. The change register shows no export changes. One noisy server was retired in week six, after the step, so it cannot explain the week-five change. Amara records each explanation, the evidence checked and her conclusion in a table.
Amara tests each alternative explanation against the evidence. She compares traffic volume, sensor uptime and calendar periods across the before and after windows, and checks whether alert volume changed on rules that were not tuned. If untuned rules stayed stable while tuned rules fell, tuning becomes the most plausible explanation.
Analyst behaviour
One more explanation is harder to test: analysts might close or escalate alerts differently once they know a study is running, which could change closure notes and therefore labels. Amara addresses this by fixing the labelled set before tuning, using records from earlier weeks, so labels cannot be influenced by the study. She also compares the share of alerts closed without notes before and after; it falls slightly, which suggests analysts write better notes now that volume is lower, a welcome side effect but one she reports as a possible bias in later label sets.
Worked example
The competing explanations table has five rows: traffic, sensor failure, retired server, export change and analyst behaviour, each with evidence and verdict.
Synthetic case. Data are synthetic.
Errors and limits
Normalising by the wrong denominator, such as hosts instead of traffic, can create or hide effects.
Exercises
- EX021 — Alternative list. List four alternative explanations for falling alerts. Rubric: 3 = four;; 2 = three;; 1 = fewer;; 0 = none.
- EX022 — Normalised result. State the normalised change. Rubric: 3 = 512 to 191 per 100 GB;; 2 = direction;; 1 = vague;; 0 = none.
- EX023 — Step timing. Why does the week-five step support the tuning explanation? Rubric: 3 = coincides with change date;; 2 = timing;; 1 = vague;; 0 = none.
- EX024 — Retired server. Why can the retired server not explain the step? Rubric: 3 = retired after the step;; 2 = timing;; 1 = vague;; 0 = none.
- EX025 — Denominator choice. Why might hosts be a poor denominator? Rubric: 3 = alerts track traffic not hosts;; 2 = mismatch;; 1 = vague;; 0 = none.
- EX026 — Evidence table. What columns does the explanations table need? Rubric: 3 = explanation, evidence, verdict;; 2 = two;; 1 = vague;; 0 = none.
Synthesis
Testing alternatives shows the tuning explanation survives, with traffic accounting for part of the raw drop.
Continue studying — chapter 04 of 08
← Chapter 03 (draft) · Course chapter index · Chapter 05 (draft) →