Chapter 30 — Laboratory II

Lesson map

Lesson 30 published. Map, example, EX088.

flowchart TD
  N0["Learning outcomes"]
  N1["30.1 Reply speed question"]
  N2["30.2 Analysis"]
  N3["Lab — Repeat reporting after fast and slow replies"]
  N4["30.3 Recommendations"]
  N5["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5
  N5 --> EX["Worked example"]
  EX --> Q["Exercise EX088"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["30.1 Reply speed question"]
  N2["30.2 Analysis"]
  N3["Lab — Repeat reporting after fast and slow replies"]
  N4["30.3 Recommendations"]
  N5["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX088"]

[Learning outcomes]
    |
    v
[30.1 Reply speed question]
    |
    v
[30.2 Analysis]
    |
    v
[Lab — Repeat reporting after fast and slow replies]
    |
    v
[30.3 Recommendations]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX088]

Learning outcomes

You will (1) analyse the effect of reply speed on later reporting; (2) present results with uncertainty; (3) recommend changes from evidence.

30.1 Reply speed question

Interviews suggested that feedback matters. In the second laboratory, Nadia analyses IT records to see whether staff who received a reply within a day were more likely to report again within three months than staff who received a late reply or none. The data are synthetic records reflecting the pattern of the year, with staff identified only by random codes. She knows the comparison is observational: people who got fast replies might differ from others, for example by working at branches near IT.

30.2 Analysis

The lab counts repeat reporting in two groups. Of 40 staff whose first report got a reply within a day, 26 reported again; of 30 who got a late reply or none, 9 reported again. Nadia computes the proportions and a Wilson interval for each. The intervals do not overlap, which suggests a real difference, though not necessarily a causal one. She notes that the result is consistent with the interviews, which strengthens her confidence that feedback matters. The intervals are 50% to 78% and 17% to 48%. She checks the counts twice.

Lab — Repeat reporting after fast and slow replies

# Repeat reporting after fast vs slow replies, with Wilson intervals (synthetic).
from math import sqrt
def wilson(k, n, z=1.96):
    p = k / n; d = 1 + z * z / n; c = (p + z * z / (2 * n)) / d; h = z * sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
    return c - h, c + h
for name, k, n in (("fast reply", 26, 40), ("slow or none", 9, 30)):
    lo, hi = wilson(k, n); print(f"{name}: {k}/{n} = {k / n:.0%} (95% CI {lo:.0%} to {hi:.0%})")

Run with python3 lab_reply.py. Verified output:

fast reply: 26/40 = 65% (95% CI 50% to 78%)
slow or none: 9/30 = 30% (95% CI 17% to 48%)

30.3 Recommendations

Nadia recommends that IT reply to every report within one working day, using short templates to save time, and that the shared mailbox be monitored by both IT staff to prevent delays during leave. She suggests checking the measure again after six months. She presents the result to managers as evidence supporting a cheap change, with honest limits about causation. Managers approve the templates, and IT agrees to track reply times as part of the quarterly review. The templates are ready within a week.

Worked example

Nadia compares repeat reporting after fast and slow replies, finds 65% versus 30% with non-overlapping intervals and recommends fast replies.

Synthetic case. Data are synthetic.

Errors and limits

Observational data cannot prove causation. Groups may differ.

Exercises

• EX088 — Group rates. What repeat rates did the two groups show?
  Show marking criteria

  Rubric: 3 = 65% after fast replies and 30% after slow or none;; 2 = 26/40 and 9/30;; 1 = vague;; 0 = none.
• EX089 — Interval result. What did the intervals show?
  Show marking criteria

  Rubric: 3 = they did not overlap;; 2 = separate;; 1 = overlap;; 0 = none.
• EX090 — Causation caution. Why can the result not prove causation?
  Show marking criteria

  Rubric: 3 = the data are observational and groups may differ;; 2 = observational;; 1 = vague;; 0 = none.

Synthesis

Observational analysis, read cautiously, can support cheap evidence-based changes.

Continue studying — chapter 30 of 40

← Chapter 29 (draft) · Course chapter index · Chapter 31 (draft) →