Chapter 29 — Measuring governance
Lesson map
Lesson 29 published. Map, example, EX085.
flowchart TD
N0["Learning outcomes"]
N1["29.1 What to measure"]
N2["29.2 A scorecard"]
N3["29.3 Distorting measures"]
N4["29.4 Reporting upward"]
N0 --> N1 --> N2 --> N3 --> N4
N4 --> EX["Worked example"]
EX --> Q["Exercise EX085"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["29.1 What to measure"]
N2["29.2 A scorecard"]
N3["29.3 Distorting measures"]
N4["29.4 Reporting upward"]
N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX085"]
[Learning outcomes]
|
v
[29.1 What to measure]
|
v
[29.2 A scorecard]
|
v
[29.3 Distorting measures]
|
v
[29.4 Reporting upward]
|
v
[Worked example] --> [Exercise EX085]
Learning outcomes
You will (1) choose measures for governance performance; (2) build a simple scorecard; (3) avoid measures that distort behaviour.
29.1 What to measure
Amara asks how Harbourline will know whether governance works. She proposes measures in three groups. Activity measures show work done, such as issues closed. Quality measures show data condition, such as rule pass rates. Outcome measures show business effects, such as fewer customer complaints about points. She notes that activity measures are easiest to collect but least meaningful, and that outcome measures matter most but are hardest to attribute to governance alone. Amara selects two measures from each group to begin with.
29.2 A scorecard
She builds a one-page scorecard for the governance group: rule pass rates for key data sets, open issues by age, share of data sets with owners, retention compliance and complaints related to data. Each measure has a target, a trend arrow and a short comment. The group reviews the scorecard monthly. She keeps it short so that discussion focuses on exceptions and actions rather than reading through long lists of figures. The first scorecard shows three measures below target, which lead to agreed actions.
29.3 Distorting measures
Measures can distort behaviour. When stewards were measured on issues closed, some closed issues without fixing them. Amara changes the measure to issues closed with verified evidence and adds the reopening rate. She explains Goodhart's observation, often summarised as when a measure becomes a target it ceases to be a good measure, and recommends using several measures together and discussing them rather than rewarding single numbers. After the change, the reopening rate falls, which suggests that stewards are now fixing problems properly before closing them in the log.
29.4 Reporting upward
Amara prepares a quarterly summary for Harbourline's board, focusing on outcomes and risks rather than detail. She includes the reduction in over-retained records, improved email completeness and fewer points complaints, with caveats about other causes. The board asks for an estimate of costs avoided. She provides a rough estimate with its assumptions stated. Dr. Lindqvist notes that boards respond to governance when it is presented in terms of risks and benefits they recognise. The board approves continued funding for the governance programme.
Worked example
Amara proposes activity, quality and outcome measures, builds a one-page monthly scorecard, fixes a distorting issues-closed measure and reports outcomes to the board with caveats.
Synthetic case. Measures are synthetic.
Errors and limits
Rewarding single numbers invites gaming; use several measures.
Exercises
- EX085 — Most meaningful. Which measure group matters most? Rubric: 3 = outcome measures;; 2 = partly correct;; 1 = other;; 0 = none.
- EX086 — Distortion fix. How was the issues-closed measure changed? Rubric: 3 = closed with verified evidence plus reopening rate;; 2 = partly correct;; 1 = partial;; 0 = none.
- EX087 — Board focus. What does the board summary focus on? Rubric: 3 = outcomes and risks;; 2 = both;; 1 = one;; 0 = none.
Continue studying — lesson 29 (chapter 29 of 40)
← Chapter 28 · Course chapter index · Chapter 30 →
Continue to the next authored lesson.