Chapter 7 — Probability and statistics
Lesson map
Lesson 07 published. Map, example, EX019.
flowchart TD
N0["Learning outcomes"]
N1["7.1 Conditional probability"]
N2["7.2 The base-rate problem"]
N3["Lab — Base rates and alert precision"]
N4["7.3 Summarising data well"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX019"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["7.1 Conditional probability"]
N2["7.2 The base-rate problem"]
N3["Lab — Base rates and alert precision"]
N4["7.3 Summarising data well"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX019"]
[Learning outcomes]
|
v
[7.1 Conditional probability]
|
v
[7.2 The base-rate problem]
|
v
[Lab — Base rates and alert precision]
|
v
[7.3 Summarising data well]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX019]
Learning outcomes
You will (1) apply conditional probability to security alerts; (2) explain the base-rate problem for detection; (3) summarise security data with appropriate statistics.
7.1 Conditional probability
Corvia's fraud detector flags suspicious taps. Its vendor states that it detects ninety-five per cent of fraudulent taps and wrongly flags only one per cent of legitimate ones. Sofia learns to ask a different question: given that a tap is flagged, how likely is it to be fraud? That is a conditional probability, and Bayes' theorem connects it to the detector's rates and to the base rate, the share of all taps that are fraudulent. Architects need this reasoning because they decide how alerts flow to analysts and whether automatic blocking is safe.
7.2 The base-rate problem
Fraud is rare: in Corvia's synthetic data, about one tap in two thousand. Sofia computes the probability that a flagged tap is fraud. The lab below shows it is under five per cent: most flags are false alarms, despite the detector's impressive-sounding rates. Automatically blocking every flagged card would therefore inconvenience many honest passengers. This is the base-rate problem, and it affects intrusion detection everywhere. Sofia recommends that flags feed a review queue and that only combinations of signals trigger blocking, a design decision that follows directly from the arithmetic.
Lab — Base rates and alert precision
# P(fraud | flag) via Bayes with synthetic rates.
base, tpr, fpr = 1/2000, 0.95, 0.01
taps = 1_000_000
fraud = taps * base; legit = taps - fraud
tp, fp = fraud * tpr, legit * fpr
print(f"taps {taps:,}: fraud {fraud:.0f}, flagged true {tp:.0f}, flagged false {fp:.0f}")
print(f"P(fraud | flag) = {tp / (tp + fp):.3f}")
Run with python3 lab_bayes.py. Verified output:
taps 1,000,000: fraud 500, flagged true 475, flagged false 9995
P(fraud | flag) = 0.045
7.3 Summarising data well
Sofia also learns to summarise. Response times for fraud reviews are skewed: most are quick, a few take days. The mean is pulled up by the slow cases, so she reports the median and the ninetieth percentile alongside it. When comparing two months, she reports counts with their denominators, not just percentages, so readers can see how much data lies behind each figure. Daniel warns that many security dashboards show averages of skewed data and percentage changes on tiny numbers, both of which mislead decision makers about what is actually happening.
Worked example
Sofia applies Bayes' theorem to vendor rates, shows most flags are false alarms at a one-in-two-thousand base rate, and redesigns the alert flow with a review queue.
Synthetic case. Detector rates and fraud data are synthetic.
Errors and limits
Vendor rates measured on other data may not hold at Corvia. Independence assumptions between signals are often false.
Exercises
- EX019 — Question reversal. Which probability matters to an analyst looking at a flagged tap? Rubric: 3 = probability of fraud given a flag;; 2 = conditional;; 1 = detection rate;; 0 = none.
- EX020 — Lab precision. What share of flagged taps were fraud in the lab? Rubric: 3 = about 4.5 per cent;; 2 = under five per cent;; 1 = 95 per cent;; 0 = none.
- EX021 — Skewed summary. Why report the median and 90th percentile for review times? Rubric: 3 = data are skewed and the mean is pulled up by slow cases;; 2 = skew;; 1 = mean only;; 0 = none.
Synthesis
Probability reasoning about base rates turns impressive detector claims into realistic alert designs.
Continue studying — chapter 07 of 50
← Chapter 06 (draft) · Course chapter index · Chapter 08 (draft) →