Chapter 40 — Verification laboratory II | Engineering Privacy and data protection

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 40 — Verification laboratory II

Lesson map

Lesson 40 published. Map, example, EX118.

flowchart TD
  N0["Learning outcomes"]
  N1["40.1 Adversarial testing"]
  N2["40.2 Fixing the statistics leak"]
  N3["40.3 Independent testing"]
  N4["40.4 Reporting to management"]
  N5["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5
  N5 --> EX["Worked example"]
  EX --> Q["Exercise EX118"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["40.1 Adversarial testing"]
  N2["40.2 Fixing the statistics leak"]
  N3["40.3 Independent testing"]
  N4["40.4 Reporting to management"]
  N5["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX118"]

[Learning outcomes]
    |
    v
[40.1 Adversarial testing]
    |
    v
[40.2 Fixing the statistics leak]
    |
    v
[40.3 Independent testing]
    |
    v
[40.4 Reporting to management]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX118]

Learning outcomes

You will (1) verify privacy properties under adversarial testing; (2) commission independent testing; (3) report verification results to management.

40.1 Adversarial testing

In the second verification laboratory Zara's class acts as adversaries against a test copy of Pellmore's platform with synthetic data. They try to link analytics data back to patients, to obtain another patient's records through the carer feature and to infer diagnoses from published statistics. Linkage fails because recipient pseudonyms differ. The carer attack fails because settings require patient authentication. The statistics attack succeeds partly for one rural clinic, where small counts were published before the differential privacy pipeline was introduced. Older outputs were never reviewed.

40.2 Fixing the statistics leak

The rural clinic leak comes from old reports still on a public page. Zara removes them and checks that the new pipeline covers all published statistics. She also searches for other legacy publications and finds two more. The finding shows that new protections do not automatically fix old outputs. She adds a rule: when a new protection is introduced, the team must search for outputs produced under the old approach and decide whether to remove, replace or keep them with justification. Here, the old table is withdrawn.

40.3 Independent testing

Pellmore commissions an independent firm to test the platform. The firm receives the same synthetic environment and scope. It finds one issue the class missed: a support tool leaks patient email addresses in an autocomplete field. Pellmore fixes it. Zara notes that independent testers bring different habits and assumptions, which is why independent assessment is valuable even after strong internal testing. She records the finding and the fix in the evidence folder alongside the class's results. The support tool is restricted to masked addresses, and a regression test is added so the leak cannot quietly return.

40.4 Reporting to management

Zara writes a verification report for Pellmore's management: what was tested, what passed, what failed and what was fixed, with remaining risks. She avoids technical detail in the summary and attaches evidence for specialists. Management asks whether the platform is secure; she answers that it passed the tests described, that two issues were fixed and that testing will continue each year. She explains that verification reduces uncertainty but cannot prove the absence of all problems, and management accepts the framing. Testing is scheduled yearly.

Worked example

Zara's class attacks the platform: linkage and carer attacks fail, a statistics attack exposes legacy rural clinic reports; an independent firm finds an autocomplete email leak; she reports results with remaining risks.

Synthetic case. Attacks run only on synthetic test copies.

Errors and limits

Passing tests do not prove absence of vulnerabilities.

Exercises

EX118 — Partial success. Which adversarial attack partly succeeded, and why? Rubric: 3 = statistics inference for a rural clinic via legacy small-count reports;; 2 = mentions rural clinic;; 1 = vague;; 0 = none.
EX119 — Legacy outputs rule. What rule did Zara add after the statistics leak? Rubric: 3 = search for outputs from the old approach whenever a new protection is introduced;; 2 = mentions legacy;; 1 = vague;; 0 = none.
EX120 — Independent value. Why commission independent testing after strong internal tests? Rubric: 3 = different habits find different issues, e.g. the autocomplete leak;; 2 = mentions independence;; 1 = unnecessary;; 0 = none.

Synthesis

Chapter 40 closes the verification cycle. Chapter 41 optimises the system.

Continue studying — chapter 40 of 50

← Chapter 39 (draft) · Course chapter index · Chapter 41 (draft) →