Chapter 50 — Final defense
Lesson map
Lesson 50 published. Map, example, EX148.
flowchart TD
N0["Learning outcomes"]
N1["50.1 Talk structure"]
N2["50.2 Likely questions"]
N3["50.3 Reflection"]
N4["50.2b After the defense"]
N5["50.3b Examiner packet"]
N6["50.1b Time discipline"]
N7["Practice note"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX148"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["50.1 Talk structure"]
N2["50.2 Likely questions"]
N3["50.3 Reflection"]
N4["50.2b After the defense"]
N5["50.3b Examiner packet"]
N6["50.1b Time discipline"]
N7["Practice note"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX148"]
[Learning outcomes]
|
v
[50.1 Talk structure]
|
v
[50.2 Likely questions]
|
v
[50.3 Reflection]
|
v
[50.2b After the defense]
|
v
[50.3b Examiner packet]
|
v
[50.1b Time discipline]
|
v
[Practice note]
|
v
[Worked example] --> [Exercise EX148]
Learning outcomes
You will (1) present the claim and evidence in 20 minutes; (2) answer safety and measurement questions; (3) state what you would not do again.
50.1 Talk structure
Claim; demo on synthetic data; verification results; one failure trial lesson; limits; questions. No live hacking demo.
Synthetic demo and verification results carry the claim; logos do not. Limits before questions set honesty as the frame. Sam's talk order matches the stem so examiners hear the failure-trial lesson before the Q&A. Logo slides are deleted in rehearsal if anyone added them from a template. Template decks from vendors are banned in the rehearsal checklist. Rehearsal checklist item one is "no vendor logos."
50.2 Likely questions
Fail-closed proof? Denominator choice? Dual-use boundary? What broken in Lab II? Practice answering with pointers to artefacts.
Artefact pointers beat improvised confidence. Fail-closed proof, denominator choice, dual-use boundary and Lab II breaks are predictable. Sam's answer cards cite checklist hashes and PulseMap export lines rather than restating slogans. Slogans without hashes are practised as failed answers in rehearsal. Failed-answer practice is recorded so the same slogan is not reused. Reused slogans from a prior failed answer cost a mark in rehearsal scoring.
50.3 Reflection
End with one design mistake and the fix — examiners trust engineers who saw their own failures.
"""Lab S: reporting attack-success rates honestly: Wilson intervals,
seed variation and a paired bootstrap. Synthetic trial outcomes only."""
import math, random
def wilson(k, n, z=1.96):
p = k/n; d = 1 + z*z/n
c = (p + z*z/(2*n))/d; h = z*math.sqrt(p*(1-p)/n + z*z/(4*n*n))/d
return c - h, c + h
for k, n in ((3, 20), (30, 200), (300, 2000)):
lo, hi = wilson(k, n); print(f"successes={k}/{n} rate={k/n:.3f} wilson95=[{lo:.3f}, {hi:.3f}]")
# five seeds of a synthetic defence evaluation: per-seed attack success rate
rates = []
for seed in range(5):
r = random.Random(seed); rates.append(sum(r.random() < 0.12 for _ in range(200))/200)
m = sum(rates)/5; sd = math.sqrt(sum((x-m)**2 for x in rates)/4)
print("per-seed rates:", " ".join(f"{x:.3f}" for x in rates), f"mean={m:.3f} sd={sd:.3f}")
# paired bootstrap: baseline vs defence on the same 300 synthetic prompts
r = random.Random(42)
base = [r.random() < 0.30 for _ in range(300)]
defn = [b and r.random() < 0.6 for b in base] # defence blocks ~40% of successes
diffs = []
for _ in range(2000):
idx = [r.randrange(300) for _ in range(300)]
diffs.append(sum(base[i] for i in idx)/300 - sum(defn[i] for i in idx)/300)
diffs.sort()
print(f"baseline_asr={sum(base)/300:.3f} defended_asr={sum(defn)/300:.3f} "
f"reduction_ci95=[{diffs[50]:.3f}, {diffs[1949]:.3f}]")
Expected output:
successes=3/20 rate=0.150 wilson95=[0.052, 0.360]
successes=30/200 rate=0.150 wilson95=[0.107, 0.206]
successes=300/2000 rate=0.150 wilson95=[0.135, 0.166]
per-seed rates: 0.110 0.130 0.135 0.105 0.145 mean=0.125 sd=0.017
baseline_asr=0.350 defended_asr=0.190 reduction_ci95=[0.120, 0.203]
Synthetic case — Riverstone Transit. Sam's defense led with the fail-closed lab, not a slide of logos. When asked for a risk score, Sam refused and showed SLA and coverage denominators.
One real design mistake and its fix builds more trust than a perfect narrative. Refusing a risk score in favour of SLA and coverage denominators is the Riverstone defense moment to emulate. Sam's mistake slide names the late poll interval and the trial that caught it. The mistake slide is required; a perfect narrative without it loses marks. Marks for the mistake slide are published in the rubric beside Lab II.
50.4 Errors and limits
Defense is not a sales pitch for a scanner vendor.
Vendor sales pitches waste examiner time and violate the measurement chapter. Defense argues from artefacts. Sam's slides contain no scanner logos; the only product names are TrackScope, FindingsDesk and PulseMap as course systems. Vendor comparisons, if any, use measured denominators from PulseMap, not brochure claims. Brochure quotes require a footnote to a PulseMap export hash. Footnotes without hashes are treated as brochure claims and removed. Sam's slide list is checked against that rule the morning of defense.
50.2b After the defense
Archive the talk deck next to the dossier with matching hashes. Note examiner requests as issues. The defense is a gate, not the end of maintenance — ownership continues (Chapter 39).
Archiving the deck with matching hashes next to the dossier keeps examiner requests actionable. Ownership continues after the gate. Sam opened three issues from examiner questions the same afternoon, still assigned to the platform squad. Issues without owners on the afternoon of defense are raised in the next standup. Standup notes link the three issues for anyone who missed the defense.
50.3b Examiner packet
Provide examiners the claim sentence, Lab II summary, failure-trial measure, and ethics paragraph one day ahead. Surprise decks waste defense time that should test your judgment.
Surprise decks burn time. Sam's packet was five pages and matched the talk order so examiners could annotate before he spoke. Packet pages are numbered to match slide numbers for annotation speed. Annotation speed matters when the panel has only five minutes to prepare. The packet carries the same version tag as the dossier, so examiners know they are reading the final text.
50.1b Time discipline
Twenty minutes means rehearsal with a timer. Cutting the limits slide to save time is the wrong cut — examiners need your honesty more than a fourth demo click.
Sam's rehearsal cut two architecture slides and kept the limits and the 45s-to-30s trial story. The trial story is timed to ninety seconds so questions still fit. Ninety seconds is rehearsed with a visible timer on the presenter's laptop. Each rehearsal records the actual time per section, and any section that runs over is cut at the next rehearsal, never the limits slide.
Practice note
After defense, schedule a maintenance review in ninety days: is the owning squad still correct, are pins current, did any corrective tickets reopen? The final chapter of the degree is the first chapter of stewardship.
Thank operators who ran synthetic trials; defense without their logs is storytelling.
Keep a parking lot for examiner ideas and process it within a week while context is fresh.
50.4b Rehearsing the defense
Two weeks before the defense, Sam gives the talk to peers who play examiners and ask the hardest questions on the likely-questions list. He times each section, cuts slides that run over, and writes short answers for the three questions he struggled with.
Peers playing examiners surface weak answers while there is still time to write short ones. Timing each part prevents a rush that drops limits. Sam's three hardest questions were dual-use, denominator and what Lab II broke; he filed one-paragraph answers before the real defense. Written answers are limited to eight lines so they can be spoken, not read as an essay. Essays are cut until they fit the eight-line card format.
Exercises
- EX148 — Outline. Six-bullet talk outline. Rubric: 3 = claim→demo→verify→failure→limits→Q; 2 = partial; 1 = vendor pitch; 0 = none.
- EX149 — Answer. 3–4 sentences answering "what's your risk score?" Rubric: 3 = refuse empty score + alternatives; 2 = partial; 1 = invents score; 0 = none.
- EX150 — Mistake. One real (synthetic-story) mistake and fix. Rubric: 3 = specific; 2 = vague; 1 = "none"; 0 = none.
Synthesis
Defend the engineering, the measures and the refusals.
Continue studying — chapter 50 of 50
← Chapter 49 (draft) · Course chapter index · Worked results ↓ · Next course via chapter index →