Chapter 8 — Defense and publication plan
Lesson map
Lesson 08 published. Map, example, EX046.
flowchart TD
N0["Learning outcomes"]
N1["8.1 The defence talk"]
N2["8.2 Anticipating questions"]
N3["8.3 Publication plan"]
N4["8.4 Impact on governance and practice"]
N5["8.6 Practice activities"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX046"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["8.1 The defence talk"]
N2["8.2 Anticipating questions"]
N3["8.3 Publication plan"]
N4["8.4 Impact on governance and practice"]
N5["8.6 Practice activities"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX046"]
[Learning outcomes]
|
v
[8.1 The defence talk]
|
v
[8.2 Anticipating questions]
|
v
[8.3 Publication plan]
|
v
[8.4 Impact on governance and practice]
|
v
[8.6 Practice activities]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX046]
Learning outcomes
You will (1) prepare a defence that presents the argument, evidence and limits in proportion; (2) anticipate and answer hostile and friendly questions; (3) plan publications, including venue choice, artefact evaluation and disclosure timing; (4) plan for impact on standards, governance and practice without overstating; and (5) define a post-doctoral research agenda.
8.1 The defence talk
A defence talk has one job: let the committee check that your claims follow from your evidence. Structure: problem and gap (10%), research questions and threat model (10%), method and baselines (20%), results with intervals and budget curves (35%), limitations and validity (15%), contributions and future work (10%). Put residual risk on a results slide, not a final "limitations" slide that runs out of time. Rehearse with a mock committee that includes someone outside your sub-field. Ana's mock defence, which lasted 45 minutes, followed this structure. Time each part in rehearsal.
8.2 Anticipating questions
Prepare written answers for predictable questions: Why this threat model and not a stronger one? Why these models? What if allowlisting is enough? What would make your result wrong? How does this hold for proprietary models you could not test? What did the red team find? What would you do differently? For each, answer in three parts: the direct answer, the evidence (slide or table), and the limit. "I don't know, and here is how I would find out" is a strong answer when true.
8.3 Publication plan
Map chapters to papers. Security venues (for example IEEE S&P, USENIX Security, ACM CCS, NDSS) often run artifact evaluation, which rewards the archive from Chapter 7; machine-learning venues often have reproducibility checklists. Choose by audience: defences intended for practitioners may also need a workshop or a technical report. Plan timing around coordinated disclosure: if a finding affects a deployed product, agree a disclosure date with the vendor before submission. Preprints speed feedback but are public — check dual-use decisions before posting. Keep an authorship and contribution statement (CRediT-style roles help).
8.4 Impact on governance and practice
Doctoral work in AI security can feed the NIST AI RMF Measure function, organisations' ISO/IEC 42001 management systems, conformity work under the EU AI Act's robustness and cybersecurity requirements, OWASP and MITRE ATLAS community resources, and open-source tooling. Contribute evidence, not slogans: an evaluation method with documented limits is more useful to a standards body than a claim that a problem is solved. Never present your result as legal advice or certification. Ana's practitioner guide is an example: it presents measured reductions and their limits.
8.5 Worked example (synthetic MAAL case)
Ana's mock defence lasted 45 minutes. A committee member asked: "Your strongest result is 5.8% residual ASR. Would you deploy this in an e-mail agent?" Her answer: "Not alone. Combined with allowlisted recipients and human confirmation for outbound mail, yes, because those controls cap impact even when injection succeeds; the thesis measures the extra reduction separation provides, not absolute safety." The external member asked about proprietary models; she showed that her method requires training access and explained that for API-only models the harness still measures ASR, but the defence cannot be applied — a scope limit that follows from the Chapter 1 system definition, which names a local open-weights model. Her plan: one paper on the harness and adaptive attacks for a security venue with artifact evaluation, one on the defence, a short practitioner guide, and a post-doctoral agenda on tool-chaining attacks and on certifying impact bounds rather than injection rates.
8.6 Practice activities
- Draft your defence outline with time proportions. 2. Write answers to the seven predictable questions. 3. Map thesis chapters to venues and a disclosure timeline. For the outline, use the proportions above and place residual risk on a results slide. For the answers, follow the three-part form and use Ana's e-mail agent answer as a model of a direct answer with a limit. For the publication map, note which venues offer artifact evaluation and where disclosure must come before submission, and give dates.
8.7 Errors and limits
Each of these errors weakens the defence or the publication:
- Spending most of the talk on background, which leaves no time to examine evidence.
- Hiding residual risk at the end, where it may be skipped when time runs out; that is why Ana showed hers with results.
- Submitting a finding about a live product before disclosure is agreed, which can breach a disclosure agreement.
- Overstating standards or regulatory impact, which is easy to refute.
- Leaving future work as "more experiments" without questions; future work should be questions, such as Ana's on tool-chaining attacks and impact bounds.
Exercises
- EX046 — Talk plan. Allocate a 40-minute defence by section. Rubric: 3 = proportions close to 8.1 with results ≥30% and limits integrated; 2 = reasonable but limits last; 1 = background-heavy; 0 = none.
- EX047 — Hard question. Answer "What would make your result wrong?" for Ana. Rubric: 3 = names concrete evidence (e.g., an adaptive family raising ASR to baseline, effect vanishing on other families, grader invalidity) and how to test; 2 = partial; 1 = vague; 0 = none.
- EX048 — Venue. Choose a venue type for the harness paper and justify. Rubric: 3 = security venue with artifact evaluation, reasons; 2 = venue only; 1 = vague; 0 = none.
- EX049 — Disclosure timing. Write a timeline for a finding affecting a deployed product. Rubric: 3 = private report, acknowledgement, agreed window, coordinated publication, safe detail level; 2 = three steps; 1 = vague; 0 = publish first.
- EX050 — Governance. Name two framework functions or clauses your work could inform, with one sentence each. Rubric: 3 = e.g., AI RMF Measure, EU AI Act Art. 15, ISO/IEC 42001 risk treatment, with accurate sentences; 2 = names only; 1 = vague; 0 = wrong.
- EX051 — Agenda. Write three post-doctoral research questions that follow from Ana's limitations. Rubric: 3 = three specific, testable questions; 2 = two; 1 = one; 0 = none.
- EX052 — Deploy? Answer the deployment question in 8.5 for your own thesis. Rubric: 3 = layered answer separating measured reduction from absolute safety, with complementary controls; 2 = partial; 1 = yes/no only; 0 = none.
Synthesis
A defence shows that claims follow from evidence; a publication plan sends that evidence where it can be checked, at a time that respects disclosure. The next research agenda grows from the limits you have honestly stated.
Continue studying — chapter 08 of 08
← Chapter 07 (draft) · Course chapter index · Worked results ↓ · Next course via chapter index →