◇ OpenCyber Academy
Technologist · Chapter 9 · English

Chapter 9 — Evidence and measurement

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 9 — Evidence and measurement

Lesson map

Lesson 09 published. Map, example, EX025.

flowchart TD
  N0["Learning outcomes"]
  N1["9.1 Metrics"]
  N2["9.2 Evidence"]
  N3["9.3 Baseline"]
  N4["9.4 Reporting"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX025"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["9.1 Metrics"]
  N2["9.2 Evidence"]
  N3["9.3 Baseline"]
  N4["9.4 Reporting"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX025"]

[Learning outcomes]
    |
    v
[9.1 Metrics]
    |
    v
[9.2 Evidence]
    |
    v
[9.3 Baseline]
    |
    v
[9.4 Reporting]
    |
    v
[Worked example] --> [Exercise EX025]

Learning outcomes

You will (1) choose privacy metrics; (2) collect evidence reliably; (3) report results.

9.1 Metrics

Aisha proposes metrics: accounts of former staff, local copies found, rights requests answered on time and CCTV footage past retention. Each has a definition and data source. She avoids metrics that are easy to count but say little, such as the number of policies published. For each metric she writes how it is calculated, who supplies the data, how often it is measured and what value would be a concern. For example, any former-staff account active for more than two days is a concern.

9.2 Evidence

She collects evidence through system reports rather than surveys where possible, because reports are more reliable. She stores evidence with dates. Surveys ask managers whether they removed leavers, but managers may remember incorrectly or answer optimistically. A system report lists the accounts directly. Where only a survey is possible, such as staff awareness, she combines it with a small test, like asking staff to show where the reporting form is, and she records how each piece of evidence was produced. Method notes prevent overclaiming.

9.3 Baseline

Aisha records a baseline: forty stale accounts, three clubs with local spreadsheets and an unknown CCTV retention position. She notes unknowns as gaps to measure. She writes down the date and method for each baseline value, because a later change in method could look like an improvement. For CCTV she plans a check at one club first. She also records two values she expects to rise at first, breach reports and access requests, explaining that better awareness can make numbers increase before risks fall.

9.4 Reporting

She reports monthly to Quayside's data protection officer. Mr. Reilly notes that measuring from the start allows progress to be shown later. Each monthly report fits on one page: a table of metrics with the current value, the previous value and the baseline, a short comment on each change and a list of actions with owners. The data protection officer forwards it to the operations director. Aisha keeps the reports in a dated folder so trends can be reviewed later. Each report names its data sources.

Worked example

Aisha defines four metrics, prefers system evidence, records a baseline including unknowns and reports monthly.

Synthetic case. Metrics are synthetic.

Errors and limits

Metrics can miss what is not measured.

Exercises

Continue studying — lesson 09 (chapter 09 of 30)

← Chapter 08 · Course chapter index · Chapter 10 →

Continue to the next authored lesson.