Chapter 12 — Probability and statistics
Lesson map
Lesson 12 published. Map, example, EX034.
flowchart TD
N0["Learning outcomes"]
N1["12.1 Measuring a scanner"]
N2["Lab — Lab — scanner precision and recall"]
N3["12.2 Lab result"]
N4["12.3 Base rates"]
N5["12.4 Thresholds"]
N6["12.5 Uncertainty"]
N7["Confidence in small samples"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX034"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["12.1 Measuring a scanner"]
N2["Lab — Lab — scanner precision and recall"]
N3["12.2 Lab result"]
N4["12.3 Base rates"]
N5["12.4 Thresholds"]
N6["12.5 Uncertainty"]
N7["Confidence in small samples"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX034"]
[Learning outcomes]
|
v
[12.1 Measuring a scanner]
|
v
[Lab — Lab — scanner precision and recall]
|
v
[12.2 Lab result]
|
v
[12.3 Base rates]
|
v
[12.4 Thresholds]
|
v
[12.5 Uncertainty]
|
v
[Confidence in small samples]
|
v
[Worked example] --> [Exercise EX034]
Learning outcomes
You will (1) compute precision and recall for a security check; (2) interpret base rates; (3) choose thresholds; (4) report uncertainty.
12.1 Measuring a scanner
After the token leak, CampusEats adds a secret scanner. Lena measures it on 200 synthetic commits whose true status she knows: 10 contain real secrets, 190 do not.
Each commit is labelled before the scanner is run, so the labels cannot be influenced by its output. The synthetic secrets imitate real formats, such as payment-provider keys, but none is valid, which lets Lena share the test set with the society without risk. Labelling first is a simple guard against measuring what you expect to find. She keeps the commits and labels in the repository so that later versions of the scanner can be measured the same way.
Lab — Lab — scanner precision and recall
"""Precision and recall of a secret scanner on 200 synthetic commits."""
tp, fp, fn = 9, 21, 1
print(f"precision {tp / (tp + fp):.2f} recall {tp / (tp + fn):.2f} flagged {tp + fp} of 200")
Run with python3 lab_scanner.py. Verified output:
precision 0.30 recall 0.90 flagged 30 of 200
12.2 Lab result
The scanner flagged 30 commits: 9 true secrets (true positives) and 21 harmless strings (false positives); it missed 1 secret (false negative). Precision is 0.30: only 30% of flags are real. Recall is 0.90: it finds 90% of real secrets.
Lena reports the full table of counts alongside the two percentages, so that readers can check the figures and see the base rate. The single missed secret was a key split across two lines, which the scanner's patterns did not expect. Lena reports the miss to the scanner's maintainers and adds a custom rule, then reruns the test set to confirm that the rule finds it without adding new false positives.
12.3 Base rates
With only 10 secrets in 200 commits, even a fairly accurate scanner produces more false than true flags. This is the base-rate effect. A scanner with low precision is tolerable if each flag takes seconds to check; otherwise developers start ignoring it, as Johnson et al. (2013) found for noisy tools.
Lena times the review of a flag at about half a minute, which the society accepts. In practice, developers see about two harmless flags for each real one. Fast review keeps them engaged, and the society tracks how often flags are dismissed without a look.
12.4 Thresholds
Raising the threshold would cut false positives but miss more secrets. For secrets, missing one is costly, so the society keeps high recall and reduces review time instead.
Lena tries a stricter setting on the same commits and records the result, so the trade-off is measured rather than guessed. She also groups repeated false positives, such as a long identifier used in test fixtures, and adds them to an allow list that a reviewer approves. That cuts review effort without lowering recall on real secrets.
12.5 Uncertainty
Ten real secrets is a small number; recall of 0.90 could easily be 0.7 or 1.0 on another sample. Lena reports counts, not just percentages.
She also notes that synthetic secrets may be easier to find than real ones, so the true recall could be lower. The society plans to re-measure after a term of real use, counting any secret found later by other means as a miss, which gives a better estimate than a single lab test. Counts let readers judge this.
Confidence in small samples
Lena explains to the society why she reports 9 of 10 rather than 90%. With ten secrets, one more miss would drop recall to 80%. Readers who see the counts understand how much the figure could move. She plans to grow the test set each term by adding newly found synthetic examples, so the estimate becomes steadier over time.
Worked example
Lena improves precision by ignoring strings in test fixture files marked as fake and by checking character variety (chapter 23). On a new set of 200 synthetic commits, flags drop to 15 with the same 9 secrets found.
Synthetic case. Counts are synthetic.
Errors and limits
Test sets made by the evaluator may be easier than real data.
Exercises
- EX034 — Lab figures. Give precision and recall. Rubric: 3 = 0.30 and 0.90; 2 = one; 1 = vague; 0 = none.
- EX035 — Base-rate effect. Why are most flags false? Rubric: 3 = few real secrets in many commits; 2 = partial; 1 = vague; 0 = none.
- EX036 — Threshold choice. Why keep high recall for secrets? Rubric: 3 = missing one is costly; 2 = partial; 1 = vague; 0 = none.
Synthesis
Measure checks with precision and recall, mindful of base rates and small samples.
Continue studying — chapter 12 of 40
← Chapter 11 (draft) · Course chapter index · Chapter 13 (draft) →