Chapter 34 — Comparative project
Lesson map
Lesson 34 published. Map, example, EX100.
flowchart TD
N0["Learning outcomes"]
N1["34.1 Choosing what to compare"]
N2["34.2 Fairness"]
N3["34.3 Trade-offs"]
N4["34.4 The extraction comparison"]
N5["34.4b Writing the comparative report"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX100"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["34.1 Choosing what to compare"]
N2["34.2 Fairness"]
N3["34.3 Trade-offs"]
N4["34.4 The extraction comparison"]
N5["34.4b Writing the comparative report"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX100"]
[Learning outcomes]
|
v
[34.1 Choosing what to compare]
|
v
[34.2 Fairness]
|
v
[34.3 Trade-offs]
|
v
[34.4 The extraction comparison]
|
v
[34.4b Writing the comparative report]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX100]
Learning outcomes
You will (1) compare two or more AI-security approaches on a shared benchmark; (2) define fair comparison conditions; (3) report trade-offs across security, utility and cost; and (4) write a comparative report.
34.1 Choosing what to compare
Pick approaches that a practitioner might really choose between: for example, a content classifier versus a tool allowlist versus a dual-model pattern for injection; or confidence rounding versus label-only outputs versus query quotas for extraction.
Rina's group chose the extraction and membership defences for NoShow's partner API because LCH was actually deciding between them, and each had a supporter on the clinic's data team. A comparison that included an approach nobody would deploy, such as shutting the API down entirely, would have been easy to win and of little use to the people making the choice.
34.2 Fairness
Give each approach the same data, tasks, attack budget and tuning effort. Use a held-out test set. Include an undefended baseline. If one approach requires more resources, report them.
In Rina's group, the categories approach was first tuned for a week while rounding was tried once with a default setting. A reviewer spotted the imbalance, and the group gave rounding the same effort, testing several precisions before comparing again. The ranking did not change, but the comparison became credible. They also reported that quotas needed a monitoring service the other two options did not.
34.3 Trade-offs
Report attack success, task success, latency and cost together. Present a table and, where useful, a plot of security against utility. No approach wins everything; the report should help a reader choose for their context.
For NoShow, the table had rows for each option and columns for membership advantage, extraction fidelity, partner usefulness and operating cost. The plot placed categories best on security but worst on partner usefulness. A partner that only needed to decide whether to send a reminder found categories enough; a research partner wanting detailed scores did not, so the report addressed both.
34.4 The extraction comparison
The extraction lab provides a small comparative result: full-precision outputs let an attacker copy a linear model exactly; two-decimal rounding barely changes fidelity; integer rounding lowers label agreement to 0.882 but still yields a useful copy. Query quotas and monitoring are therefore needed alongside output restrictions.
"""Lab X: model extraction of a toy linear scoring API by querying it.
The 'API' returns a real-valued score; d+1 well-chosen queries recover the
weights exactly, and rounding the output limits fidelity. Synthetic model."""
import random
rng = random.Random(9)
d = 5
secret_w = [round(rng.uniform(-2, 2), 3) for _ in range(d)]; secret_b = 0.42
def api(x, decimals=None):
s = sum(a*b for a, b in zip(secret_w, x)) + secret_b
return round(s, decimals) if decimals is not None else s
for dec in (None, 2, 0):
b = api([0]*d, dec)
w = [api([1 if j == i else 0 for j in range(d)], dec) - b for i in range(d)]
err = max(abs(a-c) for a, c in zip(w, secret_w))
test = [[rng.uniform(-1, 1) for _ in range(d)] for _ in range(500)]
agree = sum((sum(a*c for a, c in zip(w, x))+b > 0) == (api(x) > 0) for x in test)/500
print(f"output_rounding={str(dec):4s} queries={d+1} max_weight_error={err:.3f} label_agreement={agree:.3f}")
Expected output:
output_rounding=None queries=6 max_weight_error=0.000 label_agreement=1.000
output_rounding=2 queries=6 max_weight_error=0.004 label_agreement=0.998
output_rounding=0 queries=6 max_weight_error=0.534 label_agreement=0.882
The numbers in the lab make the point: with integer rounding the maximum weight error rises to 0.534, yet the copy still agrees with the original on 0.882 of labels after only six queries. An attacker who wants a working imitation, not the exact weights, is barely slowed, which is why a limit on the number of queries is still required.
34.4b Writing the comparative report
Open with the decision question, then the conditions of comparison, then a results table with intervals, then a trade-off discussion and a recommendation tied to context ("for partners who need scores, use rounding plus quotas; otherwise categories"). Close with limitations such as synthetic data and the attack types not tested.
Rina's group placed its recommendation on the first page, so partner managers could read it without the tables. The limitations paragraph named the synthetic NoShow data and the lack of any test against an adaptive attacker who spreads queries across several partner accounts, which quotas per partner would not catch.
34.5 Worked example (synthetic LCH case)
Rina's group compared three ways to protect NoShow's partner API: rounding probabilities to one decimal, returning only three risk categories, and per-partner quotas with monitoring. On synthetic data, categories reduced membership-inference advantage most (from 0.18 to 0.03) and extraction fidelity to 0.84 label agreement, at the cost of partners losing fine-grained scores. Quotas did not reduce per-query leakage but bounded total queries. The group recommended categories plus quotas.
The membership numbers came from repeating Lab M's method on NoShow's synthetic copy, so the reduction can be checked with the same code.
34.6 Errors and limits
- Unequal tuning effort.
- Reporting only the security metric.
- Declaring a universal winner.
Tuning one approach for a week and another for an hour makes the result reflect effort rather than design, as Rina's group nearly did. Reporting only membership advantage would have hidden that partners lose fine-grained scores under the categories option. Declaring categories the winner for every partner ignores the research partner whose work needs detailed scores; the recommendation must name the context in which it holds and the partners it suits.
Exercises
- EX100 — Fairness. List four conditions for a fair comparison. Rubric: 3 = same data/tasks, attack budget, tuning effort, held-out test (and baseline); 2 = three; 1 = one or two; 0 = none.
- EX101 — Lab reading. What label agreement does integer rounding leave, and what follows? Rubric: 3 = 0.882; useful copy remains, so add quotas/monitoring; 2 = number; 1 = vague; 0 = wrong.
- EX102 — Recommendation. Justify "categories plus quotas" in two sentences. Rubric: 3 = categories reduce per-query leakage; quotas bound volume; complementary; 2 = partial; 1 = vague; 0 = none.
Synthesis
Comparative projects teach fair evaluation and trade-off reporting — the evidence practitioners need to choose controls.
Continue studying — chapter 34 of 40
← Chapter 33 (draft) · Course chapter index · Chapter 35 (draft) →