Chapter 15 — Evidence and measurement
Lesson map
Lesson 15 published. Map, example, EX043.
flowchart TD
N0["Learning outcomes"]
N1["15.1 Defining measures"]
N2["Lab — Coverage by team"]
N3["15.2 Reading coverage"]
N4["15.3 Presenting measures"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX043"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["15.1 Defining measures"]
N2["Lab — Coverage by team"]
N3["15.2 Reading coverage"]
N4["15.3 Presenting measures"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX043"]
[Learning outcomes]
|
v
[15.1 Defining measures]
|
v
[Lab — Coverage by team]
|
v
[15.2 Reading coverage]
|
v
[15.3 Presenting measures]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX043]
Learning outcomes
You will (1) define identity measures precisely; (2) collect them consistently; (3) present them clearly to decision-makers.
15.1 Defining measures
A measure must be defined so that anyone would count it the same way. Elena defines multi-factor coverage as enabled staff accounts with at least one registered second factor, divided by all enabled staff accounts, excluding service accounts, on the first day of each month. Each part of the definition matters: including disabled accounts or service accounts would change the result. She writes definitions in a short table with the data source and known weaknesses, so the measure can be repeated by someone else.
Lab — Coverage by team
"""Multi-factor coverage by staff group from a synthetic account export (stdlib only)."""
from collections import defaultdict
rows = [("housing", 1)]*38 + [("housing", 0)]*7 + [("repairs", 1)]*22 + [("repairs", 0)]*18 + [("finance", 1)]*14 + [("finance", 0)]*1 \
+ [("office", 1)]*30 + [("office", 0)]*5 + [("admin", 1)]*4 + [("admin", 0)]*1
g = defaultdict(lambda: [0, 0])
for grp, m in rows: g[grp][0] += m; g[grp][1] += 1
for grp, (m, n) in sorted(g.items(), key=lambda x: x[1][0]/x[1][1]):
print(f"{grp:8} {m:3}/{n:<3} {100*m/n:5.1f}%")
tm = sum(v[0] for v in g.values()); tn = sum(v[1] for v in g.values()); print(f"overall {tm}/{tn} {100*tm/tn:.1f}%")
Run with python3 lab_mfa.py. Verified output:
repairs 22/40 55.0%
admin 4/5 80.0%
housing 38/45 84.4%
office 30/35 85.7%
finance 14/15 93.3%
overall 108/140 77.1%
15.2 Reading coverage
The synthetic export shows overall coverage of 108 of 140, or 77.1 percent. Teams differ sharply: finance is at 93.3 percent, office and housing in the mid-eighties, administrators at 80 percent and repairs at 55 percent. The one administrator without multi-factor is the most urgent single gap, because administrator accounts can change everything. The repairs gap is the largest in number. Elena presents both, explaining that priority depends on harm as well as size, so the IT lead can plan which to tackle first.
Fixing the one administrator account takes minutes, while the repairs gap needs devices and training, so the two fixes run on different timescales.
15.3 Presenting measures
Elena presents measures with their definition, the current value with count and base, the previous value and a short comment. She uses a simple bar chart sorted by value so the lowest team stands out. She avoids decorative charts and unnecessary decimals. Each figure is dated and linked to the export it came from. Clear, consistent presentation lets managers compare months quickly and builds trust that changes in the numbers reflect real changes at Greenhill. Each chart has a short caption explaining what changed.
Worked example
Coverage is 77.1 percent overall; one of five administrators and 18 of 40 repairs staff lack multi-factor, giving one urgent and one large gap.
Synthetic case. Teams, counts and coverage are synthetic.
Errors and limits
Including disabled or service accounts in coverage makes the measure inconsistent between months.
Exercises
- EX043 — Overall coverage. What is overall multi-factor coverage in the lab? Rubric: 3 = 108 of 140, 77.1%;; 2 = 77.1%;; 1 = 100%;; 0 = none.
- EX044 — Lowest team. Which team has the lowest coverage, and at what level? Rubric: 3 = repairs, 55.0%;; 2 = repairs;; 1 = finance;; 0 = none.
- EX045 — Urgent gap. Why is the administrator gap most urgent? Rubric: 3 = admin accounts can change everything;; 2 = power;; 1 = size;; 0 = none.
Synthesis
Precise definitions, consistent collection and clear presentation turn identity data into usable evidence.
Continue studying — chapter 15 of 40
← Chapter 14 (draft) · Course chapter index · Chapter 16 (draft) →