◇ OpenCyber Academy
Engineering · Chapter 7 · English

Chapter 7 — Probability and statistics

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 7 — Probability and statistics

Lesson map

Lesson 07 published. Map, example, EX019.

flowchart TD
  N0["Learning outcomes"]
  N1["7.1 Estimating rates"]
  N2["7.2 Base rates"]
  N3["7.3 Variation"]
  N4["7.4 Comparing groups"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX019"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["7.1 Estimating rates"]
  N2["7.2 Base rates"]
  N3["7.3 Variation"]
  N4["7.4 Comparing groups"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX019"]

[Learning outcomes]
    |
    v
[7.1 Estimating rates]
    |
    v
[7.2 Base rates]
    |
    v
[7.3 Variation]
    |
    v
[7.4 Comparing groups]
    |
    v
[Worked example] --> [Exercise EX019]

Learning outcomes

You will (1) estimate rates from security data; (2) reason about base rates in alerting; (3) quantify uncertainty.

7.1 Estimating rates

Lindiwe estimates how often Saltmarsh staff report phishing emails. From synthetic data, 180 of 1,800 staff reported at least one message in a quarter, a proportion of ten percent. She computes a confidence interval and finds it reasonably narrow because the sample is large. Prof. Hale asks her to state what the proportion describes, staff who reported at least once, so nobody mistakes it for the share of phishing emails that get reported. She reports the interval alongside the proportion. The security team uses the figure as a baseline for future awareness campaigns.

7.2 Base rates

An alert rule flags suspicious logins with ninety-nine percent sensitivity and ninety-nine percent specificity. Lindiwe calculates what happens when only one login in ten thousand is malicious. Among a million logins, about ninety-nine true attacks are flagged, along with about ten thousand false alarms. Most alerts are false even though the rule seems accurate. She learns that base rates dominate alert quality and that analysts must expect many false positives from rare-event detection. The monitoring team confirms this matches its daily experience with login alerts.

7.3 Variation

Weekly counts of malware detections vary even when nothing changes. Lindiwe models them as a Poisson process with a mean of twelve per week and shows that weeks of eighteen or more happen occasionally by chance. Managers had treated one such week as an outbreak. She recommends control limits that mark genuinely unusual weeks. Prof. Hale notes that understanding ordinary variation prevents both panic over noise and complacency about real changes. She plots a year of synthetic weekly counts with the limits drawn as horizontal lines.

7.4 Comparing groups

Saltmarsh wants to know whether depots with newer laptops have fewer malware detections. Lindiwe warns that newer laptops went to larger depots, which also have dedicated IT staff, so any difference could reflect staffing rather than hardware. She proposes comparing detection rates per device within depots of similar size. She learns that confounding is the main trap in observational security data and that comparisons need careful design. The IT manager agrees to provide device counts per depot. Lindiwe notes that even within similar depots, laptop age may still correlate with other differences she cannot measure.

Worked example

Lindiwe estimates a ten percent reporting rate, shows base rates make most alerts false, models weekly detections as Poisson and flags confounding in a laptop comparison.

Synthetic case. Figures are synthetic.

Errors and limits

An accurate-sounding detector can still produce mostly false alerts.

Exercises

Continue studying — lesson 07 (chapter 07 of 50)

← Chapter 06 · Course chapter index · Chapter 08 →

Continue to the next authored lesson.