Chapter 28 — Metrics and reporting | Technologist Identity and access management

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 28 — Metrics and reporting

Lesson map

Lesson 28 published. Map, example, EX082.

flowchart TD
  N0["Learning outcomes"]
  N1["28.1 Choosing metrics"]
  N2["28.2 Reporting"]
  N3["28.3 Using trends"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX082"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["28.1 Choosing metrics"]
  N2["28.2 Reporting"]
  N3["28.3 Using trends"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX082"]

[Learning outcomes]
    |
    v
[28.1 Choosing metrics]
    |
    v
[28.2 Reporting]
    |
    v
[28.3 Using trends]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX082]

Learning outcomes

You will (1) choose identity metrics; (2) report to non-specialists; (3) use trends to drive work.

28.1 Choosing metrics

Kai chooses a few measures that reflect real risk: percentage of staff with multi-factor authentication, percentage with phishing-resistant authenticators, leavers fully removed on time, review completion and removals, and groups without owners. Each has a clear definition and data source, so the number means the same each month. He avoids measures that only count activity, such as number of tickets, which may rise or fall without telling anyone whether risk changed. Kai checks each definition with Ruth. The definitions are written in the report glossary.

28.2 Reporting

Kai's monthly one-page report for the practice manager shows each measure, its trend over six months and a sentence explaining any change. For example: phishing-resistant authenticators rose from 12 to 41 percent after the finance and admin rollout. He uses plain words and avoids acronyms, adding a short glossary. Non-specialists need to know whether things are improving and what decision, if any, they are asked to make. The practice manager says she reads the trend line first. Kai therefore places it at the top. He keeps the report the same shape each month.

28.3 Using trends

Trends direct effort. When leaver removal on time drops in one month, Kai finds the payroll step slipped during the finance lead's holiday, and adds a deputy. When owner-less groups rise after a reorganisation, he works with managers to assign owners. Metrics are useful only when someone acts on them, so each report ends with the next month's two priorities agreed with the practice manager. The priorities are checked at the start of the next report. Kai notes whether each was met. Unmet priorities carry forward with a reason.

Worked example

Five metrics are reported monthly; phishing-resistant authenticators rise from 12% to 41% after the priority rollout.

Synthetic case. Metrics and figures are synthetic.

Errors and limits

Reporting activity counts rather than risk measures hides whether protection improved.

Exercises

EX082 — Activity metric. Why avoid ticket counts? Rubric: 3 = they do not show whether risk changed;; 2 = activity;; 1 = cost;; 0 = none.
EX083 — Rollout result. How did phishing-resistant authenticators change? Rubric: 3 = 12% to 41%;; 2 = rose;; 1 = fell;; 0 = none.
EX084 — Report ending. How does each report end? Rubric: 3 = next month's two priorities;; 2 = priorities;; 1 = vague;; 0 = none.

Synthesis

A few well-defined risk metrics, reported with trends, drive identity improvements.

Continue studying — chapter 28 of 30

← Chapter 27 (draft) · Course chapter index · Chapter 29 (draft) →