◇ OpenCyber Academy
Doctoral research · Chapter 8 · English

Chapter 8 — Defense and publication plan

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 8 — Defense and publication plan

Lesson map

Lesson 08 published. Map, example, EX050.

flowchart TD
  N0["Learning outcomes"]
  N1["8.1 Defence posture"]
  N2["8.2 Venues"]
  N3["8.3 Vendor sensitivity"]
  N4["8.4 TLP in outputs"]
  N5["8.5 Follow-on work"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX050"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["8.1 Defence posture"]
  N2["8.2 Venues"]
  N3["8.3 Vendor sensitivity"]
  N4["8.4 TLP in outputs"]
  N5["8.5 Follow-on work"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX050"]

[Learning outcomes]
    |
    v
[8.1 Defence posture]
    |
    v
[8.2 Venues]
    |
    v
[8.3 Vendor sensitivity]
    |
    v
[8.4 TLP in outputs]
    |
    v
[8.5 Follow-on work]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX050]

Learning outcomes

You will (1) prepare a defence that foregrounds limits; (2) choose publication venues and pre-print timing responsibly; (3) handle disclosure when findings could embarrass named vendors; (4) plan follow-on work that extends rather than repeats the thesis.

8.1 Defence posture

Examiners test whether you know where your result stops. Open with the claim, the evidence and the three strongest threats to validity. A candidate who raises the follow-up-bias problem before the examiner does controls the conversation.

Tomas's three threats would be follow-up bias, outcome sources that depend on the original report, and a corpus that may over-represent dramatic incidents. His sensitivity analysis answers the first, the circularity check the second, and the stated scope of the corpus limits the third.

8.2 Venues

Peer-reviewed security venues, intelligence-studies journals and practitioner conferences reach different audiences. A methods result about calibration may need both: a journal for the method and a practitioner talk for SOC leaders. Pre-prints speed dissemination but should carry the same limits section as the final paper.

For Tomas, the calibration method and codebook fit a journal; the practical message, how much weight to give "highly likely" in a vendor report when deciding to block, fits a practitioner conference for SOC leaders. The pre-print carries the same sensitivity ranges as the abstract.

8.3 Vendor sensitivity

Even with synthetic or anonymised presentation, a reader may guess which vendor a pattern describes. Offer affected parties a short preview where agreements require it, but do not let preview become veto over honest findings. Never disclose TLP-restricted content to obtain goodwill.

Tomas's handling is the model: results reported by anonymised vendor class, previews given to the two community partners whose agreements required them, one wording change accepted, and a request to drop a negative result declined. The examiner's question at the defence was answered by describing that record.

8.4 TLP in outputs

Public papers are effectively TLP:CLEAR. Anything derived from restricted inputs must be aggregated enough that the source cannot be reconstructed. Check with the sharing community when in doubt.

Tomas's results derived from the 200 TLP:AMBER+STRICT reports appear only as aggregates within vendor classes, and no report is quoted. Where a class is so small that a single report could be identified from its row, the class is merged with another before publication. The merging rule is written in the archive so that a sharing-community member can check that no restricted report is recoverable from a published row.

8.5 Follow-on work

Natural extensions: a prospective study where analysts record numeric probabilities at writing time; a cross-language comparison; an intervention that trains calibration and measures change. Each needs its own ethics file.

The prospective study answers Tomas's biggest limitation: if analysts record a numeric probability when they write a judgment, there is no need to map phrases afterwards, and the mapping assumption the examiners questioned disappears. A calibration-training intervention would study analysts as participants, so it needs their informed consent, which the current corpus-only design never required.

Worked example

At the defence, an examiner asks whether vendors were named. Tomas explains that results are reported by anonymised vendor class, that two community partners previewed the paper as their agreement required, and that one requested wording change was accepted while a request to drop a negative result was declined.

Synthetic case. The defence exchange and vendor preview are invented.

Errors and limits

Publishing a "leaderboard" of vendors' calibration from licensed data may breach licences and is methodologically shaky with small samples.

Exercises

Synthesis

Lead with limits, choose venues for audiences, let preview correct facts but not findings, respect TLP in every output, and extend rather than repeat.

Continue studying — chapter 08 of 08

← Chapter 07 (draft) · Course chapter index · Worked results ↓ · Next course via chapter index →