Chapter 4 — Privacy and data protection: competing explanations
Lesson map
Lesson 04 published. Map, example, EX021.
flowchart TD
N0["Learning outcomes"]
N1["4.1 Rival one: a simpler mix"]
N2["4.2 Rival two: more staff"]
N3["4.3 Rival three: regression and attention"]
N4["4.4 Rival four: quality trade-off"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX021"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["4.1 Rival one: a simpler mix"]
N2["4.2 Rival two: more staff"]
N3["4.3 Rival three: regression and attention"]
N4["4.4 Rival four: quality trade-off"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX021"]
[Learning outcomes]
|
v
[4.1 Rival one: a simpler mix]
|
v
[4.2 Rival two: more staff]
|
v
[4.3 Rival three: regression and attention]
|
v
[4.4 Rival four: quality trade-off]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX021]
Learning outcomes
You will (1) list rival explanations for an apparent improvement; (2) design checks for each; (3) decide which rivals the study can and cannot exclude.
4.1 Rival one: a simpler mix
If customers after launch make simpler requests, on-time rates could rise even if the portal does nothing. Hana plans to compare the share of complex requests in the two periods and to report results separately for simple and complex requests. If the portal helps, she expects improvement within both groups, not only a change in the mix. This approach follows the standard advice of stratifying by a known confounder, which means comparing like with like before drawing conclusions about the intervention, and it is easy for a reader to understand without advanced statistics.
4.2 Rival two: more staff
Brookfield might hire more privacy staff at the same time, which would also speed responses. Hana asks the data protection officer for monthly staffing figures for both periods and records any change. If staffing rises, she will note it as a rival that the study cannot separate from the portal and will say so plainly. She explains that an honest dissertation names the rivals it cannot rule out, rather than hiding them, so that Brookfield's managers know how much weight to place on her findings when deciding whether to extend the portal to other request types.
4.3 Rival three: regression and attention
The portal launch follows a period of complaints, which may have been an unusually bad stretch. Performance could improve simply because a bad period ended, a pattern called regression to the mean. Staff may also work harder because they know they are being watched. Hana collects twelve months of history before the study to check whether the pre-period was unusual, and she keeps the measurement unobtrusive by using routine ticket data rather than asking staff to log extra information, which could change their behaviour while also adding work for an already busy team.
4.4 Rival four: quality trade-off
Faster answers might be less complete. Hana's completeness check on a sample of forty answers from each period addresses this. If on-time rates rise but completeness falls, the improvement is not genuine compliance. She also tracks misdirected disclosures, which must stay at zero. Dr. Pritchard asks her to commit in advance to a rule: the portal counts as an improvement only if on-time completion rises and neither completeness nor safety worsens. Hana writes this rule into the protocol, so that a headline percentage alone cannot be presented as success.
Worked example
Hana lists four rivals (simpler mix, staffing, regression and attention, quality trade-off) and pairs each with a check: stratification, staffing records, longer history, completeness and safety outcomes.
Synthetic case. Brookfield's staffing and history are synthetic.
Errors and limits
Checks reduce but rarely remove rival explanations; unknown rivals remain.
Exercises
- EX021 — Stratify by complexity. How does Hana check the "simpler mix" rival? Rubric: 3 = compares complex share and reports results within simple and complex groups;; 2 = mentions complexity;; 1 = ignores mix;; 0 = none.
- EX022 — Staffing rival. What will Hana do if staffing rises at launch? Rubric: 3 = report it as an unseparable rival;; 2 = mentions staff;; 1 = ignores it;; 0 = none.
- EX023 — Regression to mean. What is regression to the mean in this study? Rubric: 3 = improvement because an unusually bad period ended;; 2 = mentions chance;; 1 = confuses with trend;; 0 = none.
- EX024 — Unobtrusive data. Why use routine ticket data rather than extra logging? Rubric: 3 = extra logging could change staff behaviour and add work;; 2 = mentions observation;; 1 = omits;; 0 = none.
- EX025 — Improvement rule. State Hana's pre-committed improvement rule. Rubric: 3 = on-time rises and neither completeness nor safety worsens;; 2 = two parts;; 1 = one part;; 0 = none.
- EX026 — Unknown rivals. Why can listing rivals never be complete? Rubric: 3 = some explanations are unknown to the researcher;; 2 = mentions limits;; 1 = claims completeness;; 0 = none.
Synthesis
Chapter 4 arms the design against rival explanations. Chapter 5 writes the proposal and checks sample size.
Continue studying — chapter 04 of 08
← Chapter 03 (draft) · Course chapter index · Chapter 05 (draft) →