Chapter 29 — Measuring security
Lesson map
Lesson 29 published. Map, example, EX085.
flowchart TD
N0["Learning outcomes"]
N1["29.1 Choosing measures"]
N2["29.2 Misleading metrics"]
N3["29.3 Using lab results"]
N4["29.4 Reporting trends"]
N0 --> N1 --> N2 --> N3 --> N4
N4 --> EX["Worked example"]
EX --> Q["Exercise EX085"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["29.1 Choosing measures"]
N2["29.2 Misleading metrics"]
N3["29.3 Using lab results"]
N4["29.4 Reporting trends"]
N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX085"]
[Learning outcomes]
|
v
[29.1 Choosing measures]
|
v
[29.2 Misleading metrics]
|
v
[29.3 Using lab results]
|
v
[29.4 Reporting trends]
|
v
[Worked example] --> [Exercise EX085]
Learning outcomes
You will (1) choose meaningful security measures; (2) avoid misleading metrics; (3) report trends to leaders.
29.1 Choosing measures
Aisha helps Ashfield choose security measures for the board. She selects measures linked to the top risks: share of critical vulnerabilities fixed within target, leavers removed within one day, staff reporting simulated phishing, backups restored successfully in tests and incidents by type. Each measure has a definition, data source, owner and target. Dr. Carrow notes that measures should help decisions, so each must connect to a risk or objective the board cares about. The board approves the five measures for quarterly reporting.
29.2 Misleading metrics
Some metrics mislead. Counting blocked attacks at the firewall produces large numbers that look impressive but say little about risk. Counting training completions shows attendance, not behaviour. Aisha explains these limits to the IT manager and replaces them with measures linked to outcomes. She learns that metrics which always look good or always rise without meaning should be treated with suspicion. The IT manager agrees and removes the firewall count from the board report, keeping it only as an internal operational figure for his team where it still has some value.
29.3 Using lab results
Aisha includes results from her earlier labs as baselines: ten active leaver accounts found in the access review, 89 of 400 vulnerability findings overdue and 2.0 percent of synthetic member passwords found in a breached list. She sets targets for each and plans to measure again in six months. She notes that baselines make improvement visible, and that without them nobody can tell whether security is getting better. The IT manager agrees with the targets and assigns each measure an owner who will report progress at the quarterly security meeting.
29.4 Reporting trends
Aisha designs a one-page quarterly report for the board: each measure with its current value, target, trend arrow and a short comment. Measures off target include a planned action and owner. The board can see at a glance which risks are improving. Dr. Carrow notes that trends matter more than single values, because a single quarter can be unusual while a trend shows direction. The board chair says the new report takes five minutes to read and gives a clearer picture than the old twelve-page report did.
Worked example
Aisha links measures to top risks, replaces vanity metrics, uses lab results as baselines and designs a one-page trend report.
Synthetic case. Measures are synthetic.
Errors and limits
Counting blocked attacks looks impressive but says little about risk.
Exercises
- EX085 — Vanity metric. Which metric says little about risk? Rubric: 3 = blocked attacks at the firewall;; 2 = partly correct;; 1 = other;; 0 = none.
- EX086 — Leaver baseline. What is the leaver baseline? Rubric: 3 = ten active leaver accounts;; 2 = partly correct;; 1 = wrong;; 0 = none.
- EX087 — Report format. What does the board report show for each measure? Rubric: 3 = value, target, trend and comment;; 2 = partly correct;; 1 = vague;; 0 = none.
Continue studying — lesson 29 (chapter 29 of 40)
← Chapter 28 · Course chapter index · Chapter 30 →
Continue to the next authored lesson.