Chapter 3 — Original contribution proposal
Lesson map
Lesson 03 published. Map, example, EX015.
flowchart TD
N0["Learning outcomes"]
N1["3.1 The design"]
N2["3.2 Outcomes and sample"]
N3["3.3 Ethical and practical risks"]
N4["3.4 Committee review"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX015"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["3.1 The design"]
N2["3.2 Outcomes and sample"]
N3["3.3 Ethical and practical risks"]
N4["3.4 Committee review"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX015"]
[Learning outcomes]
|
v
[3.1 The design]
|
v
[3.2 Outcomes and sample]
|
v
[3.3 Ethical and practical risks]
|
v
[3.4 Committee review]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX015]
Learning outcomes
You will (1) write a doctoral proposal with a staggered design; (2) justify sample size; (3) anticipate ethical and practical risks.
3.1 The design
Mireille proposes a stepped rollout across Vantor's twelve departments. All departments start on the existing programme. Every two months, three randomly chosen departments switch to the reporting-focused programme: a one-click report button, an automatic thank-you within minutes, a weekly digest showing what reported emails turned out to be, and short monthly simulations at a fixed Phish Scale difficulty. Randomising the order of switching lets her compare departments before and after while controlling for seasonal changes in email threats that affect everyone at once.
3.2 Outcomes and sample
The primary outcome is the share of real suspicious emails reported by at least one recipient within thirty minutes, coded as in chapter 1. Secondary outcomes are simulation report rates, simulation click rates and staff-reported burden from a short survey. Because departments, not individuals, are randomised, she accounts for clustering: staff in one department share managers and email patterns. With twelve clusters, she estimates the study can detect a difference of about ten percentage points, and she says plainly that smaller effects may go undetected.
3.3 Ethical and practical risks
Mireille lists risks and responses. Staff may feel watched: the works council reviews all messages, and only department-level results are reported. Managers may ask for names of people who click: the data agreement forbids it. A real attack during the study could be mistaken for a simulation: simulated emails carry a hidden marker known to the security team, and any real report is handled normally. The IT team may change the email filter mid-study: changes are logged so the analysis can account for them.
3.4 Committee review
Her proposal committee includes a statistician, an organisational psychologist and a practitioner from another council. The statistician asks for a permutation test that respects the stepped design; the psychologist asks how burden will be measured; the practitioner warns that report buttons can overwhelm small security teams. Mireille adds an automated triage step that groups duplicate reports. The committee approves with these changes. Proposals improve most when reviewers from different disciplines can challenge them before data collection begins. The committee's comments and her responses are kept in an appendix to the proposal for later readers.
Worked example
Mireille proposes a randomised stepped rollout across twelve departments, defines outcomes with clustering, plans ethical safeguards and revises after a multidisciplinary committee.
Synthetic case. The design and departments are synthetic; the Phish Scale is real.
Errors and limits
Twelve clusters give limited power. Stepped designs can be disrupted by organisational changes.
Exercises
- EX015 — Randomised order. Why randomise the order in which departments switch? Rubric: 3 = to control for seasonal threat changes affecting everyone;; 2 = seasonality;; 1 = vague;; 0 = none.
- EX016 — Primary outcome. What is the study's primary outcome? Rubric: 3 = share of real suspicious emails reported within thirty minutes;; 2 = reporting;; 1 = clicks;; 0 = none.
- EX017 — Clustering reason. Why account for clustering? Rubric: 3 = staff in a department share managers and email patterns;; 2 = shared context;; 1 = vague;; 0 = none.
- EX018 — Real attack confusion. How does the protocol avoid confusing real attacks with simulations? Rubric: 3 = simulated emails carry a hidden marker known to the security team;; 2 = marker;; 1 = vague;; 0 = none.
- EX019 — Practitioner warning. What did the practitioner warn about? Rubric: 3 = report buttons can overwhelm small security teams;; 2 = workload;; 1 = vague;; 0 = none.
- EX020 — Triage response. How did Mireille respond to the workload concern? Rubric: 3 = an automated triage step grouping duplicate reports;; 2 = triage;; 1 = vague;; 0 = none.
Synthesis
A strong proposal randomises what it can, measures what matters, plans for ethical risks and invites challenge before data collection.
Continue studying — chapter 03 of 08
← Chapter 02 (draft) · Course chapter index · Chapter 04 (draft) →