Chapter 12 — Probability and statistics
Lesson map
Lesson 12 published. Map, example, EX024.
flowchart TD
N0["Learning outcomes and connection"]
N1["12.1 Rules you will use constantly"]
N2["12.2 Conditional probability and Bayes"]
N3["12.3 Binomial reasoning for redundancy"]
N4["12.4 Describing data"]
N5["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX024"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["12.1 Rules you will use constantly"]
N2["12.2 Conditional probability and Bayes"]
N3["12.3 Binomial reasoning for redundancy"]
N4["12.4 Describing data"]
N5["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX024"]
[Learning outcomes and connection]
|
v
[12.1 Rules you will use constantly]
|
v
[12.2 Conditional probability and Bayes]
|
v
[12.3 Binomial reasoning for redundancy]
|
v
[12.4 Describing data]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX024]
Learning outcomes and connection
After this chapter you can (1) apply the addition, complement and multiplication rules and say when events are independent; (2) use Bayes' theorem to interpret an alert, including the effect of the base rate; (3) compute the chance of at least one failure over repeated trials and the chance that enough of $n$ redundant units are available; and (4) summarise skewed data with medians and percentiles. It builds on Chapter 1 and prepares Chapters 15, 27 and 29.
12.1 Rules you will use constantly
Probabilities are numbers between 0 and 1. The complement rule: $P(\text{not } A) = 1 - P(A)$. For independent events, $P(A \text{ and } B) = P(A)P(B)$. For any events, $P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$ — adding without subtracting the overlap double-counts. "At least one" problems are easiest by complement: if a backup fails silently on a given day with probability $p$, independently each day, the chance of at least one silent failure in $n$ days is $1 - (1-p)^n$. Independence is an assumption, often false: two generators fed from the same fuel tank fail together. Such common-cause failures are why redundancy is weaker than the multiplication rule suggests.
12.2 Conditional probability and Bayes
$P(A \mid B)$ is the probability of $A$ given that $B$ occurred. Bayes' theorem relates the two directions: $$P(\text{malicious} \mid \text{alert}) = \frac{P(\text{alert} \mid \text{malicious})\,P(\text{malicious})}{P(\text{alert})}.$$ A detector with a 95% true-positive rate and 1% false-positive rate sounds excellent. If only 1 event in 1,000 is malicious, most alerts are still false: the base rate dominates. This is the most important statistical idea in security operations, and it explains why analysts drown in alerts.
12.3 Binomial reasoning for redundancy
If $n$ identical units are each available with probability $1-p$, independently, the number available follows a binomial distribution: $P(k \text{ available}) = \binom{n}{k}(1-p)^k p^{n-k}$. A pumping station that needs at least 10 of its 12 pumps can compute how often it meets demand.
For the 12-pump station in Lab 6, with each pump unavailable 10 % of the time, the sum over 10, 11 and 12 available pumps gives 0.889. A shared maintenance crew or a single power feed breaks the independence assumption, so the real figure is lower. Adding a thirteenth pump raises the result to about 0.966, which is the kind of comparison a capital request needs.
12.4 Describing data
Times to detect and recover from incidents are skewed: most are short, a few are very long. The mean is pulled up by the long tail; the median (middle value) is not; percentiles (the 90th percentile, p90) describe the tail. Report median and p90 together, with the sample size (Chapter 29, Lab 12).
Lab 12 shows the effect: its mean time to detect is 27.2 hours while the median is 4.0.
For a board, the sentence "half of incidents were detected within 4 hours; one in ten took more than about 56" carries more information than any single average.
12.5 Worked example and Lab 6 (synthetic)
This case is synthetic. Calder Mutual's monitoring service sees 20,000 events a day; analysts estimate 1 in 1,000 is malicious; the detector's rates are those above. CW's nightly telemetry backup fails silently with probability 0.002 per day. A pumping station has 12 pumps each unavailable 10% of the time.
# Lab 6 - base rates, alarm volume and repeated trials (synthetic, offline)
from math import comb
prior, tpr, fpr = 0.001, 0.95, 0.01
ppv = tpr * prior / (tpr * prior + fpr * (1 - prior))
print(f"P(malicious | alert) = {ppv:.3f}")
events_per_day = 20_000
print(f"expected false alerts/day = {events_per_day * (1 - prior) * fpr:.0f}; true = {events_per_day * prior * tpr:.0f}")
p_day = 0.002 # chance per day that a given backup fails silently
for days in (30, 365):
print(f"P(at least one silent backup failure in {days} days) = {1 - (1 - p_day) ** days:.3f}")
n, p = 12, 0.1 # 12 independent pumps, each unavailable 10% of the time
need = 10
print(f"P(at least {need} of {n} available) = {sum(comb(n, k) * (1-p)**k * p**(n-k) for k in range(need, n+1)):.3f}")
Expected output:
P(malicious | alert) = 0.087
expected false alerts/day = 200; true = 19
P(at least one silent backup failure in 30 days) = 0.058
P(at least one silent backup failure in 365 days) = 0.518
P(at least 10 of 12 available) = 0.889
Fewer than 1 alert in 11 is real; analysts face about 200 false alerts a day. A "rare" daily backup failure is more likely than not over a year — which is why restore drills, not backup logs, are the evidence that matters. And a station needing 10 of 12 pumps falls short about 11% of the time under these assumptions: if pump outages share a cause (one maintenance crew, one power feed), it will be worse.
12.6 Common errors and limits
- Ignoring the base rate (the "95% accurate" fallacy).
- Multiplying probabilities of events that share a cause.
- Reporting a mean for skewed times without the median and tail.
- Treating estimated inputs as exact; the outputs inherit their uncertainty.
Lab 6 gives each a number: 95 % detection still yields a 0.087 chance that an alert is real; two generators sharing fuel fail together far more often than 0.05 squared (EX024); the 27.2-hour mean detection time in Lab 12 describes no incident; and the 0.002 daily backup failure rate is itself an estimate, so the 0.518 yearly figure inherits its uncertainty.
Exercises — chapter 12
EX023 — Change the base rate. Objective: see how prior and false-positive rate drive alert value. Inputs: Lab 6. Steps: compute $P(\text{malicious}\mid\text{alert})$ for prior 0.01 and, separately, for prior 0.001 with FPR 0.001. Explain which change helps more and what it means operationally. Expected evidence: two values and an explanation. Rubric: 3 = ≈ 0.490 and ≈ 0.487 with an explanation that lowering FPR (tuning, enrichment) is the lever analysts control; 2 = values only; 1 = one value; 0 = none.
EX024 — Common cause. Objective: quantify the effect of shared causes. Inputs: two backup generators each failing to start with probability 0.05; with probability 0.02 a shared cause (contaminated common fuel) fails both. Steps: compute the probability both fail assuming independence, then with the common cause (approximate as $0.02 + 0.05^2$), and comment. Expected evidence: two numbers and a comment. Rubric: 3 = 0.0025 versus ≈ 0.0225, roughly nine times worse, and a remedy (separate fuel, fuel testing); 2 = numbers only; 1 = one number; 0 = none.
Chapter synthesis
Probability rules are simple; their assumptions are not. Base rates decide whether an alert is worth reading, "rare per day" becomes "likely per year", redundancy is weakened by common causes and skewed data needs medians and percentiles. State independence assumptions and test what happens when they fail.
Continue studying — chapter 12 of 40
← Chapter 11 · Course chapter index · Chapter 13 →
Continue to the next authored lesson.