Chapter 4 — Data governance: competing explanations
Lesson map
Lesson 04 published. Map, example, EX021.
flowchart TD
N0["Learning outcomes"]
N1["4.1 Why rivals matter"]
N2["4.2 Designing checks"]
N3["4.3 The culture explanation"]
N4["4.4 Evidence that would weaken the hypothesis"]
N0 --> N1 --> N2 --> N3 --> N4
N4 --> EX["Worked example"]
EX --> Q["Exercise EX021"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["4.1 Why rivals matter"]
N2["4.2 Designing checks"]
N3["4.3 The culture explanation"]
N4["4.4 Evidence that would weaken the hypothesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX021"]
[Learning outcomes]
|
v
[4.1 Why rivals matter]
|
v
[4.2 Designing checks]
|
v
[4.3 The culture explanation]
|
v
[4.4 Evidence that would weaken the hypothesis]
|
v
[Worked example] --> [Exercise EX021]
Learning outcomes
You will (1) list rival explanations for an expected effect; (2) design checks for each; (3) decide what evidence would weaken the main hypothesis.
4.1 Why rivals matter
Tomás expects stewarded departments to fix errors faster. Dr. Solberg asks what else could produce that result. He lists rivals: a new address lookup tool introduced at the same time, a fall in the number of errors reported so staff have more time per error, more experienced staff moving into the stewarded departments, and a Hawthorne effect where staff work harder because they know they are observed. Shadish, Cook and Campbell describe such threats to internal validity, and naming them in advance prevents convenient interpretation later.
4.2 Designing checks
For each rival, Tomás designs a check. The lookup tool was deployed in all five departments, so it affects both groups and is removed by the design, but he confirms deployment dates. Error volume is measured per department per month. Staff experience is checked with anonymised tenure figures from human resources at department level. The Hawthorne effect is harder to test, so he plans interviews asking how work changed and compares the first two months with later months, expecting any novelty effect to fade.
4.3 The culture explanation
Peers had suggested that culture matters more than stewardship. Tomás operationalises culture modestly: whether departments already shared corrections informally before the pilot. Interviews and written summaries from managers indicate that waste collection already phoned council tax about changes, while libraries did not. He realises this could inflate the waste department's apparent improvement if informal sharing increased. He adds a department-level comparison within the stewarded group to see whether both stewarded departments improve similarly or only the one with prior collaboration. Both are compared.
4.4 Evidence that would weaken the hypothesis
Tomás writes down what results would count against stewardship: improvement concentrated in months with low error volume, improvement equal in both groups, or improvement only in the department with prior informal sharing. He commits to reporting such results honestly. Dr. Solberg explains that a hypothesis is useful only if some evidence could contradict it, and that writing those conditions down protects researchers from the temptation to explain away awkward findings after months of investment in a project. These conditions enter the analysis plan.
Worked example
Tomás lists four rivals, designs checks for each, operationalises culture as prior informal sharing and states in advance which results would weaken the stewardship hypothesis.
Synthetic case. The rivals and checks are synthetic; Shadish, Cook and Campbell (2002) is a real source on validity threats.
Errors and limits
Some rivals cannot be fully ruled out; report them as limits.
Exercises
- EX021 — Four rivals. Name two rival explanations. Rubric: 3 = new tool, lower error volume, experienced staff, Hawthorne effect;; 2 = two;; 1 = one;; 0 = none.
- EX022 — Tool rival. Why does the design remove the lookup tool rival? Rubric: 3 = deployed in all departments;; 2 = partly correct;; 1 = vague;; 0 = none.
- EX023 — Hawthorne check. How will Tomás check the Hawthorne effect? Rubric: 3 = interviews and comparing early with later months;; 2 = mentions months;; 1 = vague;; 0 = none.
- EX024 — Culture measure. How is culture operationalised? Rubric: 3 = prior informal sharing of corrections;; 2 = partly correct;; 1 = vague;; 0 = none.
- EX025 — Contrary evidence. Name one result that would weaken the hypothesis. Rubric: 3 = improvement only in low-volume months, equal in both groups, or only where sharing existed;; 2 = one;; 1 = vague;; 0 = none.
- EX026 — Falsifiability. Why write contrary conditions in advance? Rubric: 3 = to avoid explaining away awkward results;; 2 = mentions bias;; 1 = vague;; 0 = none.
Continue studying — lesson 04 (chapter 04 of 08)
← Chapter 03 · Course chapter index · Chapter 05 →
Continue to the next authored lesson.