Chapter 1 — Advanced research integrity and methods

Learning outcomes

You will (1) set integrity rules for security research involving a real-world partner; (2) choose methods that suit organisational security questions; (3) plan ethics review and data handling.

1.1 The candidate and the question

Yusuf Al-Rashid begins a two-year doctoral research track in information security at the fictional Marlow University, supervised by Prof. Ingeborg Hald. His partner organisation is Fernvale Municipal Services, a fictional city council with thirty departments and about four thousand staff. His question is whether ranking security controls by assessed risk, rather than applying them uniformly, shortens the time to patch critical vulnerabilities and reduces excess access rights. All data in this book are synthetic, generated to mirror the structure of such a study.

1.2 Integrity in security research

Prof. Hald sets integrity rules at the first meeting. Yusuf will never test live systems without written authorisation, will report any vulnerability he encounters to Fernvale's security team rather than publishing it, will pre-register hypotheses before analysing outcome data and will keep raw data separate from analysis code. She explains that security research carries a special risk: a careless paper can hand attackers a map of an organisation's weaknesses, so publication must describe patterns without exposing exploitable details of the partner. Yusuf signs them.

1.3 Methods for organisational questions

Yusuf compares methods. Controlled experiments are rarely possible inside a council, because departments cannot be randomly assigned to weaker security. Case studies give depth but weak generalisation. Quasi-experimental designs, discussed by Shadish, Cook and Campbell, compare groups that adopt a practice with groups that do not, before and after adoption. He chooses a difference-in-differences design, because Fernvale is rolling out risk-ranked patching in some departments first, creating a natural comparison group whose trends can absorb citywide changes. Prof. Hald approves the choice after discussion.

1.4 Ethics and data handling

The study uses logs of patch dates, vulnerability severity ratings and account permission records. These include staff identifiers, so Yusuf applies to the university ethics committee with a plan: identifiers are replaced by keyed pseudonyms inside Fernvale before export, only department-level results are published and raw extracts are destroyed after the thesis is examined. The committee approves with one condition: Fernvale's security lead must review any publication for operational sensitivity before submission. Yusuf records the condition in his data management plan. Yusuf agrees.

Worked example

Yusuf frames a question on risk-ranked controls at Fernvale, sets integrity rules on authorisation, disclosure and pre-registration, chooses difference-in-differences and obtains ethics approval with pseudonymised exports and a security-lead review condition.

Synthetic case. Marlow, Fernvale and all people are synthetic; Shadish, Cook and Campbell (2002) is real.

Errors and limits

Publishing partner-specific weaknesses turns research into an attack guide; report patterns instead.

Exercises

• EX001 — Research question. What does Yusuf's question compare?
  Show marking criteria

  Rubric: 3 = risk-ranked versus uniform control application;; 2 = partly correct;; 1 = partial;; 0 = none.
• EX002 — Partner scale. How many departments does Fernvale have?
  Show marking criteria

  Rubric: 3 = thirty;; 2 = partly correct;; 1 = wrong;; 0 = none.
• EX003 — Live testing rule. When may Yusuf test live systems?
  Show marking criteria

  Rubric: 3 = only with written authorisation;; 2 = partly correct;; 1 = vague;; 0 = none.
• EX004 — Found vulnerability. What happens if Yusuf finds a vulnerability?
  Show marking criteria

  Rubric: 3 = report it to Fernvale's security team;; 2 = partly correct;; 1 = publish;; 0 = none.
• EX005 — Chosen design. Which design does Yusuf choose?
  Show marking criteria

  Rubric: 3 = difference-in-differences;; 2 = partly correct;; 1 = other;; 0 = none.
• EX006 — Natural comparison. Why does a comparison group exist?
  Show marking criteria

  Rubric: 3 = risk-ranked patching rolls out to some departments first;; 2 = partly correct;; 1 = vague;; 0 = none.
• EX007 — Ethics condition. What condition did the ethics committee add?
  Show marking criteria

  Rubric: 3 = security lead reviews publications for sensitivity;; 2 = partly correct;; 1 = vague;; 0 = none.