Chapter 7 — Probability and statistics
Lesson map
Lesson 07 published. Map, example, EX019.
flowchart TD
N0["Learning outcomes"]
N1["7.1 Rates and Wilson intervals"]
N2["7.2 Base rates and precision"]
N3["7.3 Ranking defences"]
N4["7.3b Reporting uncertainty to non-statisticians"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX019"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["7.1 Rates and Wilson intervals"]
N2["7.2 Base rates and precision"]
N3["7.3 Ranking defences"]
N4["7.3b Reporting uncertainty to non-statisticians"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX019"]
[Learning outcomes]
|
v
[7.1 Rates and Wilson intervals]
|
v
[7.2 Base rates and precision]
|
v
[7.3 Ranking defences]
|
v
[7.3b Reporting uncertainty to non-statisticians]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX019]
Learning outcomes
You will (1) report a rate with an interval, not a bare percentage; (2) apply Bayes' rule to alarm precision; (3) refuse a ranking that the sample cannot support.
7.1 Rates and Wilson intervals
Three successes in twenty trials is 15%. A Wilson 95% interval for that is about 5%–36%. You can say the rate is probably not zero and probably below 40%; you cannot say it is better than a defence that scored 20/100 (20%). Larger n shrinks the interval: 300/2,000 is still 15% but about 13.5%–16.6%. Albrecht and Jensen warn against empty single scores; an interval is the cheapest honesty.
"""Lab S: reporting attack-success rates honestly: Wilson intervals,
seed variation and a paired bootstrap. Synthetic trial outcomes only."""
import math, random
def wilson(k, n, z=1.96):
p = k/n; d = 1 + z*z/n
c = (p + z*z/(2*n))/d; h = z*math.sqrt(p*(1-p)/n + z*z/(4*n*n))/d
return c - h, c + h
for k, n in ((3, 20), (30, 200), (300, 2000)):
lo, hi = wilson(k, n); print(f"successes={k}/{n} rate={k/n:.3f} wilson95=[{lo:.3f}, {hi:.3f}]")
# five seeds of a synthetic defence evaluation: per-seed attack success rate
rates = []
for seed in range(5):
r = random.Random(seed); rates.append(sum(r.random() < 0.12 for _ in range(200))/200)
m = sum(rates)/5; sd = math.sqrt(sum((x-m)**2 for x in rates)/4)
print("per-seed rates:", " ".join(f"{x:.3f}" for x in rates), f"mean={m:.3f} sd={sd:.3f}")
# paired bootstrap: baseline vs defence on the same 300 synthetic prompts
r = random.Random(42)
base = [r.random() < 0.30 for _ in range(300)]
defn = [b and r.random() < 0.6 for b in base] # defence blocks ~40% of successes
diffs = []
for _ in range(2000):
idx = [r.randrange(300) for _ in range(300)]
diffs.append(sum(base[i] for i in idx)/300 - sum(defn[i] for i in idx)/300)
diffs.sort()
print(f"baseline_asr={sum(base)/300:.3f} defended_asr={sum(defn)/300:.3f} "
f"reduction_ci95=[{diffs[50]:.3f}, {diffs[1949]:.3f}]")
Expected output:
successes=3/20 rate=0.150 wilson95=[0.052, 0.360]
successes=30/200 rate=0.150 wilson95=[0.107, 0.206]
successes=300/2000 rate=0.150 wilson95=[0.135, 0.166]
per-seed rates: 0.110 0.130 0.135 0.105 0.145 mean=0.125 sd=0.017
baseline_asr=0.350 defended_asr=0.190 reduction_ci95=[0.120, 0.203]
The same fifteen percent means very different things at n equals twenty and at n equals two thousand. An interval makes that visible without a statistics lecture. Sam reports the interval next to every attack-success rate so a defence is not crowned on a tiny sample. The lab's seed variation and paired bootstrap give two further checks: stability across seeds and a difference that may include zero.
7.2 Base rates and precision
A detector with 95% true-positive rate and 5% false-positive rate looks excellent until the base rate of the bad class is 0.1%. Then almost every alert is a false positive (precision ≈ 1.9%). Bayes' rule: precision = TPR·π / (TPR·π + FPR·(1−π)). PulseMap must show precision at the client's base rate, not only TPR.
"""Lab B: base rates and alarm precision (Bayes' rule).
A detector for malicious prompts with given true- and false-positive rates
is applied to traffic with different shares of malicious prompts."""
for tpr, fpr in ((0.95, 0.05), (0.95, 0.01), (0.80, 0.001)):
for prev in (0.10, 0.01, 0.001):
tp = tpr * prev; fp = fpr * (1 - prev)
prec = tp / (tp + fp)
per_10k = round(10000 * (tp + fp))
print(f"tpr={tpr:.2f} fpr={fpr:.3f} prevalence={prev:.3f} precision={prec:.3f} alerts_per_10k={per_10k}")
Expected output:
tpr=0.95 fpr=0.050 prevalence=0.100 precision=0.679 alerts_per_10k=1400
tpr=0.95 fpr=0.050 prevalence=0.010 precision=0.161 alerts_per_10k=590
tpr=0.95 fpr=0.050 prevalence=0.001 precision=0.019 alerts_per_10k=509
tpr=0.95 fpr=0.010 prevalence=0.100 precision=0.913 alerts_per_10k=1040
tpr=0.95 fpr=0.010 prevalence=0.010 precision=0.490 alerts_per_10k=194
tpr=0.95 fpr=0.010 prevalence=0.001 precision=0.087 alerts_per_10k=109
tpr=0.80 fpr=0.001 prevalence=0.100 precision=0.989 alerts_per_10k=809
tpr=0.80 fpr=0.001 prevalence=0.010 precision=0.890 alerts_per_10k=90
tpr=0.80 fpr=0.001 prevalence=0.001 precision=0.445 alerts_per_10k=18
Precision collapses when the bad class is rare, which is exactly PulseMap's setting. A dashboard that shows only true-positive rate will look excellent while flooding analysts with false alarms. Computing precision at the client's measured prevalence, as the lab table does, is what Sam puts in the report. If prevalence is unknown, he shows a small table across a range rather than picking a flattering number.
7.3 Ranking defences
Two defences with overlapping intervals are not ranked. Prefer a paired design on the same prompts and a bootstrap of the difference, as the lab shows. If the interval for the difference includes zero, report "no significant difference in this sample", not a winner.
Synthetic case — Riverstone Transit. A vendor claimed "our filter is 99% effective". Sam asked for the base rate and sample size; neither was in the brochure. The pilot used 200 synthetic prompts at a 1% attack share and measured precision ≈ 16%.
7.4 Errors and limits
Independence of trials fails when prompts share a template. Intervals assume a fixed process; changing the model mid-test voids them. Never invent a p-value for a table you did not test.
Shared templates make trials dependent, so intervals that assume independence are too narrow. Changing the model mid-test mixes two processes and voids the sample. Sam never attaches a significance claim to a table that was not tested for one, and he restarts the sample when the defence under test changes.
7.3b Reporting uncertainty to non-statisticians
A detection rate of 7 out of 10 and a rate of 700 out of 1,000 print the same percentage but deserve very different confidence. When Sam reports PulseMap coverage to the Riverstone Transit security manager, he gives the count, the interval and one plain sentence: "we saw seven of ten test events; the true rate could plausibly be anywhere from about 40% to 90%." Managers then ask for more trials instead of treating a small sample as a fact, and the ranking of defences waits until intervals stop overlapping.
Exercises
- EX019 — Interval. State what you can claim from 8/40 successes at 95%. Rubric: 3 = Wilson ≈ 10.5%–34.5% (or Clopper–Pearson), cannot rank vs 25%; 2 = interval only; 1 = point estimate; 0 = wrong.
- EX020 — Precision. Detector TPR 0.9, FPR 0.02, π = 0.005. Precision? Rubric: 3 ≈ 0.184 with working; 2 = one slip; 1 = formula; 0 = none.
- EX021 — Ranking refusal. Two filters score 12/100 and 18/100. Write the sentence that refuses a ranking. Rubric: 3 = cites overlapping uncertainty / need for paired design; 2 = vague caution; 1 = picks a winner; 0 = none.
Synthesis
Statistics is the language for not overclaiming. Rates with intervals and precision at the real base rate keep PulseMap honest.
Continue studying — chapter 07 of 50
← Chapter 06 (draft) · Course chapter index · Chapter 08 (draft) →