Chapter 12 — Probability and statistics
Lesson map
Lesson 12 published. Map, example, EX034.
flowchart TD
N0["Learning outcomes"]
N1["12.1 Skewed data"]
N2["Lab — Lab — remediation statistics"]
N3["12.2 Lab result"]
N4["12.3 Interpreting the interval"]
N5["12.4 Sample size"]
N6["12.5 Comparing periods"]
N7["12.6 Presenting"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX034"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["12.1 Skewed data"]
N2["Lab — Lab — remediation statistics"]
N3["12.2 Lab result"]
N4["12.3 Interpreting the interval"]
N5["12.4 Sample size"]
N6["12.5 Comparing periods"]
N7["12.6 Presenting"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX034"]
[Learning outcomes]
|
v
[12.1 Skewed data]
|
v
[Lab — Lab — remediation statistics]
|
v
[12.2 Lab result]
|
v
[12.3 Interpreting the interval]
|
v
[12.4 Sample size]
|
v
[12.5 Comparing periods]
|
v
[12.6 Presenting]
|
v
[Worked example] --> [Exercise EX034]
Learning outcomes
You will (1) summarise remediation times; (2) distinguish mean and median; (3) estimate a proportion with an interval; (4) interpret uncertainty.
12.1 Skewed data
Times to fix findings are skewed: most are quick, a few take months. The median describes the typical case; the mean is pulled up by long delays.
In the lab's data, a few findings took more than a month, which drags the mean well above the median. Reporting only the mean would suggest that a typical finding takes nearly three weeks, when half are fixed in under two. Both numbers are true, but they answer different questions, and Samuel reports both.
Lab — Lab — remediation statistics
"""Remediation times and a 95% interval for the share fixed within 14 days (stdlib only)."""
import math, statistics
days = [2, 3, 3, 5, 6, 7, 8, 9, 10, 12, 13, 15, 18, 21, 25, 30, 33, 41, 55, 70]
k = sum(d <= 14 for d in days); n = len(days); p = k / n
se = math.sqrt(p * (1 - p) / n)
print(f"n {n}, mean {statistics.mean(days):.1f}, median {statistics.median(days)}, within 14 days {k}/{n} = {p:.2f}")
print(f"approx 95% interval {max(0,p-1.96*se):.2f}-{min(1,p+1.96*se):.2f}")
Run with python3 lab_stats.py. Verified output:
n 20, mean 19.3, median 12.5, within 14 days 11/20 = 0.55
approx 95% interval 0.33-0.77
12.2 Lab result
Across 20 synthetic findings, the mean time to fix is 19.3 days and the median 12.5. Eleven of 20 (0.55) were fixed within the 14-day target. An approximate 95% interval for that proportion is 0.33 to 0.77.
The two averages differ by almost a week, which is the effect of the long tail. Eleven of twenty fixed on time sounds like a coin toss, but the interval shows how little twenty findings can tell us. Samuel's report puts the count, the rate and the interval in one sentence so readers see all three together.
12.3 Interpreting the interval
With only 20 findings, the true on-time rate could plausibly be anywhere from about a third to three quarters. Claims like "we fix 55% on time" should carry this uncertainty.
Samuel's report to the cloud team says the on-time rate is "about half, with wide uncertainty", which is accurate and does not invite either alarm or complacency. When findings accumulate over the year, the interval will narrow and the rate can be compared with the target more confidently. He also notes that the interval assumes findings are independent.
12.4 Sample size
Quadrupling the sample roughly halves the interval width. Samuel plans to report quarterly once more findings accumulate.
To halve the width from about forty-four points to twenty-two, the team would need around eighty findings, which a quarter's weekly checks should provide. Waiting for more data is better than reacting to noise. Samuel notes the planned sample size in the report so readers know when firmer conclusions will be possible. Until then, he reports the count and the interval rather than a confident rate, and he explains to the team why a larger sample matters before any target is judged met or missed.
12.5 Comparing periods
Differences between periods need intervals too; a rise from 55% to 65% on small samples may be noise.
If next quarter shows 65% on time, the team will want to claim improvement. Samuel's plan is to compute intervals for both periods and check whether they overlap substantially. If they do, the report will say the change is not yet clear, and suggest looking at the long tail instead, where causes are easier to see. A change in what the checker detects can also shift the numbers, since new checks often find old, slow-to-fix problems. Samuel records checker changes beside each period so comparisons stay fair.
12.6 Presenting
Charts show medians and percentiles, with sample sizes labelled.
Samuel's chart shows each quarter's median time to fix as a point, with a bar from the twenty-fifth to the seventy-fifth percentile, and the number of findings written beneath. A chart without sample sizes would let a quarter with five findings look as reliable as one with eighty. Labels name the units, days, and the target line at fourteen days is drawn across the chart, so readers can see at once how each quarter compares. Samuel avoids three-dimensional effects and decorative colours, which make values harder to read without adding information.
Why the median matters here
The median of 12.5 days says a typical finding is fixed within target. The mean of 19.3 days is pulled up by four findings that took 33 to 70 days. Looking at those four, Samuel finds two waited for a vendor patch and two had no clear owner. Fixing ownership is cheap and addresses half the tail. This kind of analysis is more useful to the team than a single average, because it points to causes they can act on.
Worked example
The cloud team had planned to report "45% of findings late" as a crisis. Samuel shows the wide interval and the median of 12.5 days, below target. The team reframes the report around the long tail of findings over 30 days, which have specific causes.
Synthetic case. Data are synthetic.
Errors and limits
The simple interval used here is approximate; for small samples better methods exist.
Exercises
- EX034 — Lab figures. What were mean, median and on-time share? Rubric: 3 = 19.3, 12.5, 0.55; 2 = two; 1 = one; 0 = none.
- EX035 — Interval. What was the interval? Rubric: 3 = 0.33-0.77; 2 = partial; 1 = vague; 0 = none.
- EX036 — Reframing. How was the report reframed? Rubric: 3 = focus on long tail over 30 days; 2 = partial; 1 = vague; 0 = none.
Synthesis
Report typical values and uncertainty honestly.
Continue studying — chapter 12 of 40
← Chapter 11 (draft) · Course chapter index · Chapter 13 (draft) →