Chapter 12 — Probability and statistics
Lesson map
Lesson 12 published. Map, example, EX034.
flowchart TD
N0["Learning outcomes"]
N1["12.1 Normal behaviour varies"]
N2["12.2 Mean and standard deviation"]
N3["Lab — unusual outbound mail"]
N4["12.3 Reading the result"]
N5["12.4 Thresholds"]
N6["12.6 Reporting"]
N7["Seasonality"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX034"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["12.1 Normal behaviour varies"]
N2["12.2 Mean and standard deviation"]
N3["Lab — unusual outbound mail"]
N4["12.3 Reading the result"]
N5["12.4 Thresholds"]
N6["12.6 Reporting"]
N7["Seasonality"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX034"]
[Learning outcomes]
|
v
[12.1 Normal behaviour varies]
|
v
[12.2 Mean and standard deviation]
|
v
[Lab — unusual outbound mail]
|
v
[12.3 Reading the result]
|
v
[12.4 Thresholds]
|
v
[12.6 Reporting]
|
v
[Seasonality]
|
v
[Worked example] --> [Exercise EX034]
Learning outcomes
You will (1) describe distributions of user activity; (2) use means, standard deviations and z-scores to find unusual behaviour; (3) understand base rates and false alarms; (4) report uncertainty.
12.1 Normal behaviour varies
Each user sends a different number of emails per day. A library staff account sends about 40; most students send a handful. Unusual means unusual for that account.
For s2047, a student who sent a few messages a day, 212 in a day was obviously strange; for the library account, 40 was routine. A single threshold across all accounts would either miss the student or constantly flag the library. Comparing each account with its own history removes most of that problem, though not all.
12.2 Mean and standard deviation
The mean is the typical value; the standard deviation measures spread. A z-score — (today − mean) ÷ standard deviation — shows how many spreads today's value lies from typical.
Lucas computes both from the previous seven days for each account. For an account that sends between two and three messages a day, the standard deviation is small, so a large jump produces a very high z-score. For an account with wide daily swings, the same jump produces a modest score, which is what we want.
Lab — unusual outbound mail
"""Flag accounts whose outbound mail today is far above their own history (synthetic)."""
import statistics
history = {"s1001": [4, 6, 5, 3, 7, 5, 4], "s2047": [2, 3, 1, 2, 4, 2, 3], "staff_lib": [40, 38, 45, 42, 39, 41, 44]}
today = {"s1001": 8, "s2047": 212, "staff_lib": 47}
for u, h in history.items():
m, s = statistics.mean(h), statistics.stdev(h)
z = (today[u] - m) / s
print(f"{u:9} mean={m:5.1f} sd={s:4.1f} today={today[u]:4} z={z:7.1f} {'FLAG' if z > 4 else ''}")
Run with python3 lab_zscore.py. Verified output:
s1001 mean= 4.9 sd= 1.3 today= 8 z= 2.3
s2047 mean= 2.4 sd= 1.0 today= 212 z= 214.7 FLAG
staff_lib mean= 41.3 sd= 2.6 today= 47 z= 2.2
12.3 Reading the result
s2047 normally sends about 2.4 messages a day; today it sent 212, a z-score of about 215 — far beyond any normal variation. s1001 (z = 2.3) and the library account (z = 2.2) are slightly above typical but well within ordinary days.
The size of s2047's score reflects how steady its history was: with only small daily variation, even a modest rise would score highly, and 212 messages score enormously. The two near-threshold accounts show the opposite case, small rises on accounts whose activity already varies, which is why they are not flagged.
12.4 Thresholds
Lucas flags z above 4. With hundreds of thousands of accounts, a threshold of 2 would flag thousands every day; base rates matter, as Axelsson (2000) showed for intrusion detection.
At four, Riverton's daily screening flags a handful of accounts, which two staff and six students can review. Lucas checked that s2047 would have been flagged at any threshold up to two hundred, so the choice did not affect this case. A lower threshold would have buried it among routine variation and tired the analysts.
12.5 Limits of z-scores
Seven days is a short history; z-scores assume roughly symmetric variation, which counts rarely have. They are a screening tool, not proof.
New accounts have no history at all, and the start of term changes everyone's behaviour at once. A student who sends coursework to a whole class on one day can produce a high score innocently. Lucas treats a flag as a reason to look at an account, combined with other signs, never as a finding on its own.
12.6 Reporting
Lucas reports values with history length and threshold so readers know how the flag was made.
His summary line for s2047 reads: 212 messages today against a seven-day mean of about 2.4, z about 215, threshold 4. A reader can see at once how extreme the value is and how short the history was. Without those details, a flag would be impossible to compare with flags produced by other rules or other analysts. Dr. Kim asked for the same format in every screening report, so trends across weeks could be compared directly.
Seasonality
Activity changes with the academic calendar: exam weeks, holidays and term starts. A history that spans a holiday understates normal term-time activity. Lucas compares like with like where possible, using the same week type in previous terms.
Worked example
Dr. Kim asks whether s2047 could be a legitimate bulk send, such as a society newsletter. Lucas checks recipients: 212 different addresses, none in the student's past contacts, all with the same "Library fines" subject. Combined with the risky sign-in, he classifies it as compromise.
Synthetic case. Values come from the lab script.
Errors and limits
Accounts with no history cannot be scored; new users need a different approach.
Exercises
- EX034 — z-score meaning. Define a z-score in this context. Rubric: 3 = (today − mean) ÷ sd: spreads from typical; 2 = partial; 1 = vague; 0 = none.
- EX035 — Lab flags. Which account was flagged and why were others not? Rubric: 3 = s2047 z≈215; others z≈2.3, 2.2 below threshold 4; 2 = partial; 1 = wrong; 0 = none.
- EX036 — Threshold reasoning. Why not flag at z above 2? Rubric: 3 = many accounts → thousands of daily false flags (base rate); 2 = partial; 1 = no reason; 0 = none.
Synthesis
Statistics compare behaviour with each account's own normal; thresholds must respect base rates and short histories.
Continue studying — chapter 12 of 40
← Chapter 11 (draft) · Course chapter index · Chapter 13 (draft) →