Chapter 42 — Data governance: applied research
Lesson map
Lesson 42 published. Map, example, EX124.
flowchart TD
N0["Learning outcomes"]
N1["42.1 The question"]
N2["42.2 Study design"]
N3["42.3 Interpreting results"]
N4["42.4 Reporting"]
N0 --> N1 --> N2 --> N3 --> N4
N4 --> EX["Worked example"]
EX --> Q["Exercise EX124"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["42.1 The question"]
N2["42.2 Study design"]
N3["42.3 Interpreting results"]
N4["42.4 Reporting"]
N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX124"]
[Learning outcomes]
|
v
[42.1 The question]
|
v
[42.2 Study design]
|
v
[42.3 Interpreting results]
|
v
[42.4 Reporting]
|
v
[Worked example] --> [Exercise EX124]
Learning outcomes
You will (1) frame a research question from governance practice; (2) design a study with synthetic data; (3) report findings with limits.
42.1 The question
Kofi's applied research asks whether reconciliation breaks predict errors in district leakage estimates. If they do, break counts could serve as an early warning before figures are published. He frames the question precisely: in synthetic district data, is the number of reconciliation breaks in a month associated with the size of later corrections to that district's leakage figure? He notes that the question is useful to Tidewater because it would let stewards focus checking effort where errors are likely. Prof. Varga approves it.
42.2 Study design
Kofi generates synthetic data for forty districts over twenty-four months, with breaks and corrections linked through a known relationship plus noise. He fits a simple linear relationship using Python's statistics module and checks whether the estimated slope recovers the relationship he built in. He explains that testing methods on synthetic data with known answers shows whether the analysis can find a real effect before applying it to real data, where the true answer is unknown. He fixes the random seed so results repeat exactly.
42.3 Interpreting results
The analysis recovers a positive association close to the value built in, with wide variation between districts. Kofi cautions that recovering a planted effect only shows the method works, not that real districts behave this way. He outlines a plan to apply the method to real anonymised district data under Tidewater's approval, with Shadish, Cook and Campbell's discussion of validity guiding which alternative explanations, such as district size, must be checked. He also notes that the strength of association in real data may be much weaker than in the planted example.
42.4 Reporting
Kofi writes a short paper with the question, synthetic design, results, limits and a plan for real data. He shares code so others can reproduce the synthetic results. Prof. Varga notes that a clear statement of what the study can and cannot show is the most valuable part of applied research for practitioners, who will otherwise read more into results than they support, especially when results confirm what they already wanted to believe. The paper is presented at a departmental seminar, where questions sharpen the limits section.
Worked example
Kofi asks whether reconciliation breaks predict leakage corrections, tests the method on synthetic districts with a planted relationship, recovers it, states limits and plans a real-data study.
Synthetic case. Data are synthetic; Shadish, Cook and Campbell (2002) is real.
Errors and limits
Recovering a planted effect does not prove a real one.
Exercises
- EX124 — Research question. What did Kofi ask? Rubric: 3 = whether breaks predict later leakage corrections;; 2 = partly correct;; 1 = vague;; 0 = none.
- EX125 — Synthetic purpose. Why use synthetic data with known answers? Rubric: 3 = to check the method can find a real effect;; 2 = partly correct;; 1 = vague;; 0 = none.
- EX126 — Alternative explanation. Name one alternative explanation to check. Rubric: 3 = district size;; 2 = partly correct;; 1 = vague;; 0 = none.
Continue studying — lesson 42 (chapter 42 of 50)
← Chapter 41 · Course chapter index · Chapter 43 →
Continue to the next authored lesson.