Chapter 12 — Probability and statistics

Lesson map

Lesson 12 published. Map, example, EX034.

flowchart TD
  N0["Learning outcomes"]
  N1["12.1 Summaries"]
  N2["12.2 Small samples"]
  N3["12.3 Trends and causes"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX034"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["12.1 Summaries"]
  N2["12.2 Small samples"]
  N3["12.3 Trends and causes"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX034"]

[Learning outcomes]
    |
    v
[12.1 Summaries]
    |
    v
[12.2 Small samples]
    |
    v
[12.3 Trends and causes]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX034]

Learning outcomes

You will (1) summarise identity measurements with medians and ranges; (2) reason about chance in small samples; (3) avoid overreading trends.

12.1 Summaries

Elena measures how many days pass between a leaver's last day and their account being disabled. The values vary widely: many are zero or one day, a few are weeks. The mean is pulled up by the long delays, so she reports the median as the typical case and the maximum to show the worst. She also counts how many exceed Greenhill's target of one day. Choosing summaries that match the shape of the data prevents a misleading single number from hiding the problem cases.

12.2 Small samples

Greenhill has about ten leavers a month, so monthly figures jump around by chance. If six were disabled on time one month and eight the next, that may not mean improvement. Elena groups three months together before comparing, and reports counts rather than only percentages. She explains to managers that with small numbers, a single slow case changes the percentage a lot, so they should look for patterns over several months rather than reacting to every change. She also shows the actual counts on every chart.

12.3 Trends and causes

When figures improve after a change, it is tempting to credit the change. Elena checks for other explanations: fewer leavers that month, a different person processing them or a holiday period. She marks when changes were made on her charts and compares periods of similar length. Where she cannot rule out alternatives, she says the improvement coincided with the change rather than claiming it was caused by it. Careful wording keeps her reports honest and her later capstone evaluation credible.

Worked example

Over a quarter of 30 synthetic leavers, median delay is 1 day, maximum 31 days, and 14 exceed the one-day target.

Synthetic case. Leavers and delays are synthetic.

Errors and limits

Reacting to one month's change in a small team treats chance variation as a real trend.

Exercises

• EX034 — Median choice. Why report the median leaver delay rather than the mean?
  Show marking criteria

  Rubric: 3 = long delays pull the mean up;; 2 = skew;; 1 = simpler;; 0 = none.
• EX035 — Grouping months. Why group three months together?
  Show marking criteria

  Rubric: 3 = small monthly numbers vary by chance;; 2 = chance;; 1 = tidier;; 0 = none.
• EX036 — Cautious wording. What wording does Elena use when alternatives cannot be ruled out?
  Show marking criteria

  Rubric: 3 = improvement coincided with the change;; 2 = coincided;; 1 = caused;; 0 = none.

Synthesis

Medians, maximums, grouped periods and careful causal wording make small identity datasets meaningful.

Continue studying — chapter 12 of 40

← Chapter 11 (draft) · Course chapter index · Chapter 13 (draft) →