Chapter 42 — Human factors and cyber policy: applied research | Engineering Human factors and cyber policy

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 42 — Human factors and cyber policy: applied research

Lesson map

Lesson 42 published. Map, example, EX124.

flowchart TD
  N0["Learning outcomes"]
  N1["42.1 Framing the question"]
  N2["42.2 Ethical design"]
  N3["42.3 Sharing findings"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX124"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["42.1 Framing the question"]
  N2["42.2 Ethical design"]
  N3["42.3 Sharing findings"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX124"]

[Learning outcomes]
    |
    v
[42.1 Framing the question]
    |
    v
[42.2 Ethical design]
    |
    v
[42.3 Sharing findings]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX124]

Learning outcomes

You will (1) frame an applied research question from operational data; (2) design an ethical study within the programme; (3) share findings responsibly.

42.1 Framing the question

Operating data raise a research question: does adding a short sentence about why an email was dangerous to the verdict message improve later reporting accuracy, compared with a plain verdict? Aiko frames the question precisely, defines reporting accuracy as the share of a person's reports that are genuine threats, and chooses a six-month horizon. She checks published work and finds that feedback content is little studied compared with training content, so the question could add useful knowledge beyond Greyford. Prof. Kerr approves the question.

42.2 Ethical design

Aiko designs a study that randomises teams, not individuals, to receive either the plain or the explained verdict, so that colleagues share one version. Both versions are harmless and the change is small. She seeks approval from the university ethics committee and agreement from staff representatives, informs staff that feedback wording is being compared and allows any team to opt out. Data are analysed at team level only. The design follows the same principles as the simulation rules: respect, transparency and no individual consequences.

42.3 Sharing findings

After six months, the explained verdict teams show slightly higher reporting accuracy, but the difference is within the range expected by chance given the number of teams. Aiko reports the result honestly as inconclusive and suggests a larger multi-organisation study. She shares findings with staff first, then with the regulator and finally in a short paper with synthetic illustrations rather than raw data. Prof. Kerr reminds her that inconclusive results are worth publishing, because unpublished null results distort the research record.

Worked example

Aiko frames a feedback wording question, randomises teams with ethics approval and reports an inconclusive result openly.

Synthetic case. The study is synthetic.

Errors and limits

Few teams give low power. Contamination between teams is possible.

Exercises

EX124 — Unit of randomisation. What unit was randomised? Rubric: 3 = teams;; 2 = team;; 1 = individuals;; 0 = none.
EX125 — Accuracy definition. How was reporting accuracy defined? Rubric: 3 = share of a person's reports that are genuine threats;; 2 = genuine share;; 1 = vague;; 0 = none.
EX126 — Null result. Why publish the inconclusive result? Rubric: 3 = unpublished nulls distort the research record;; 2 = record;; 1 = vague;; 0 = none.

Synthesis

Applied research turns operational questions into ethical studies and reports results honestly, including nulls.

Continue studying — chapter 42 of 50

← Chapter 41 (draft) · Course chapter index · Chapter 43 (draft) →