Chapter 7 — Probability and statistics
Lesson map
Lesson 07 published. Map, example, EX019.
flowchart TD
N0["Learning outcomes"]
N1["7.1 Rates with uncertainty"]
N2["Lab — Report rates with Wilson intervals by team"]
N3["7.2 Small groups"]
N4["7.3 Honest reporting"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX019"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["7.1 Rates with uncertainty"]
N2["Lab — Report rates with Wilson intervals by team"]
N3["7.2 Small groups"]
N4["7.3 Honest reporting"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX019"]
[Learning outcomes]
|
v
[7.1 Rates with uncertainty]
|
v
[Lab — Report rates with Wilson intervals by team]
|
v
[7.2 Small groups]
|
v
[7.3 Honest reporting]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX019]
Learning outcomes
You will (1) estimate rates with uncertainty; (2) avoid misleading comparisons of small groups; (3) report results without identifying individuals.
7.1 Rates with uncertainty
Greyford's last simulation found that nine of sixty field engineers reported the email within an hour. Aiko computes the rate, fifteen percent, and a confidence interval to show how uncertain it is with only sixty people. The interval is wide, so a later result of twenty percent might not be a real improvement. She learns that every rate from a small group should carry its uncertainty, otherwise managers will react to random variation as if it were a trend. She adds the interval to every chart.
Lab — Report rates with Wilson intervals by team
# Report rates with Wilson 95% intervals by team (synthetic simulation results).
import math
def wilson(k, n, z=1.96):
p = k / n; d = 1 + z * z / n; c = (p + z * z / (2 * n)) / d
h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
return c - h, c + h
teams = {"field engineers": (9, 60), "office": (31, 140), "treatment": (12, 45), "lab": (4, 8), "fleet": (6, 30)}
for t, (k, n) in teams.items():
lo, hi = wilson(k, n); flag = " (under 20: merge before reporting)" if n < 20 else ""
print(f"{t:15} {k}/{n} = {k/n:.1%} 95% CI {lo:.1%} to {hi:.1%}{flag}")
Run with python3 lab_rates.py. Verified output:
field engineers 9/60 = 15.0% 95% CI 8.1% to 26.1%
office 31/140 = 22.1% 95% CI 16.1% to 29.7%
treatment 12/45 = 26.7% 95% CI 16.0% to 41.0%
lab 4/8 = 50.0% 95% CI 21.5% to 78.5% (under 20: merge before reporting)
fleet 6/30 = 20.0% 95% CI 9.5% to 37.3%
7.2 Small groups
Aiko compares report rates across five teams. The smallest team, with eight people, shows the highest rate, but its interval is very wide. She warns against ranking teams publicly, because small teams swing randomly and rankings shame people for chance results. She also notes that publishing a rate for a team of eight could identify individuals. Her rule is to report only groups of twenty or more, merging small teams where necessary. The lab marks the eight-person team for merging, and she explains to the team leader why their result will not be shown separately.
7.3 Honest reporting
Aiko writes the results for management: overall reporting rate with its interval, change since last quarter, and a note that differences between teams are mostly within uncertainty. She includes the Phish Scale difficulty of the lure, as described by Dawkins and Jacobs (2023), because a harder lure would naturally lower reporting. Statistics in human factors work must be honest about uncertainty and context so that people are not judged by noise. She also lists what the figures cannot show, such as whether staff who did not report simply never opened the email, so managers do not overread the numbers.
Worked example
Aiko estimates report rates with Wilson intervals, warns against ranking small teams, merges groups under twenty and reports with lure difficulty.
Synthetic case. Teams and counts are synthetic; Dawkins and Jacobs (2023) is real.
Errors and limits
Intervals assume independent responses. Merging groups hides real differences.
Exercises
- EX019 — Field engineer interval. What interval did the lab give for 9 of 60? Rubric: 3 = 15.0%, Wilson 95% interval 8.1% to 26.1%;; 2 = wide;; 1 = vague;; 0 = none.
- EX020 — Ranking risk. Why not rank teams publicly by report rate? Rubric: 3 = small teams swing randomly and rankings shame chance results;; 2 = noise;; 1 = vague;; 0 = none.
- EX021 — Group size rule. What minimum group size does Aiko report? Rubric: 3 = twenty;; 2 = 20;; 1 = eight;; 0 = none.
Synthesis
Rates from small groups must carry uncertainty and context, and must never single out individuals.
Continue studying — chapter 07 of 50
← Chapter 06 (draft) · Course chapter index · Chapter 08 (draft) →