## Chapter 12 — Probability and statistics ### Lesson map Lesson 12 published. Map, example, EX034. ```mermaid flowchart TD N0["Learning outcomes"] N1["12.1 Sampling for audit"] N2["12.2 Confidence intervals"] N3["Lab — Audit sample and confidence interval"] N4["12.3 Common errors"] N5["Synthesis"] N0 --> N1 --> N2 --> N3 --> N4 --> N5 N5 --> EX["Worked example"] EX --> Q["Exercise EX034"] ``` ### Logical drawing ```mermaid flowchart LR N0["Learning outcomes"] N1["12.1 Sampling for audit"] N2["12.2 Confidence intervals"] N3["Lab — Audit sample and confidence interval"] N4["12.3 Common errors"] N5["Synthesis"] N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX034"] ``` ```text [Learning outcomes] | v [12.1 Sampling for audit] | v [12.2 Confidence intervals] | v [Lab — Audit sample and confidence interval] | v [12.3 Common errors] | v [Synthesis] | v [Worked example] --> [Exercise EX034] ``` ### Learning outcomes You will (1) estimate a proportion from an audit sample; (2) compute and interpret a confidence interval; (3) avoid common statistical errors in security reports. ### 12.1 Sampling for audit Fernway has about four thousand user and service accounts across systems. Hannah cannot review every account, so Lucas draws a random sample of two hundred and checks each for problems such as missing owners or stale access. He finds twenty-six with problems. The sample proportion is thirteen per cent. Random sampling matters: if he had checked only accounts he already suspected, the estimate would be biased upward. He records how the sample was drawn, with the random seed, so the audit can be repeated and checked by someone else. ### 12.2 Confidence intervals A single proportion hides uncertainty. Lucas computes a ninety-five per cent confidence interval using the Wilson method, which behaves well for proportions. The lab below shows the estimate and interval, and the implied range of affected accounts across the population. He reports that between roughly nine and eighteen per cent of accounts likely have problems, about 360 to 740 accounts. That range is more honest and more useful than thirteen per cent alone, because it tells Hannah how much clean-up effort to plan for. ### Lab — Audit sample and confidence interval ```python # Wilson 95% interval for an account-audit sample (synthetic). import math N, n, x, z = 4000, 200, 26, 1.96 p = x / n den = 1 + z*z/n; centre = (p + z*z/(2*n)) / den half = z * math.sqrt(p*(1-p)/n + z*z/(4*n*n)) / den lo, hi = centre - half, centre + half print(f"sample {x}/{n} = {p:.1%}; Wilson 95% CI {lo:.1%} to {hi:.1%}") print(f"implied affected accounts: {lo*N:.0f} to {hi*N:.0f} of {N}") ``` Run with `python3 lab_sample.py`. Verified output: ```text sample 26/200 = 13.0%; Wilson 95% CI 9.0% to 18.4% implied affected accounts: 361 to 735 of 4000 ``` ### 12.3 Common errors Lucas lists errors he has seen in security reports: percentages without counts, comparisons of small numbers presented as dramatic changes, averages of skewed data and confusing correlation with cause. A report claiming phishing clicks fell by half after training, based on four clicks falling to two, says little. He resolves to always report counts with proportions, intervals where sampling is involved and the method used. Statistics do not make a report more scientific unless used carefully; misused, they lend false authority to weak claims. ### Worked example Lucas samples 200 of 4,000 accounts, finds 26 with problems, computes a Wilson interval and reports a range of affected accounts. > **Synthetic case.** Accounts and findings are synthetic. ### Errors and limits Intervals assume random sampling; convenience samples invalidate them. Problems found depend on reviewer consistency. ### Exercises - **EX034 — Why random.** Why must the audit sample be random? *Rubric:* 3 = targeted samples bias the estimate;; 2 = bias;; 1 = faster;; 0 = none. - **EX035 — Lab interval.** What interval did the lab report? *Rubric:* 3 = about 9.0 to 18.4 per cent, roughly 361 to 735 accounts;; 2 = interval;; 1 = vague;; 0 = none. - **EX036 — Four to two.** Why is phishing clicks fell by half, four to two, weak evidence? *Rubric:* 3 = tiny counts make the change meaningless by chance;; 2 = small numbers;; 1 = strong;; 0 = none. ### Synthesis Sampling with stated methods and intervals turns partial audits into honest estimates. ### Continue studying — chapter 12 of 40 [← Chapter 11 (draft)](#ch-11) · [Course chapter index](../SUBJECTS.md) · [Chapter 13 (draft) →](#ch-13) ## Worked answers **EX034.** Because checking only suspected accounts would bias the estimate upward. **EX035.** About 9.0% to 18.4%, implying roughly 361 to 735 affected accounts. **EX036.** Because with such small counts the change could easily be chance.