OpenCyber Academy
Doctoral research · Chapter 3 · English
Course code: OCA-196Chapter 03: OCA-196-C003

Chapter 3 — Original contribution proposal

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 03 published. Map, example, EX015.

flowchart TD
  N0["Learning outcomes"]
  N1["3.1 Hypotheses"]
  N2["3.2 Design"]
  N3["3.3 Rings"]
  N4["3.4 Threats to validity"]
  N5["3.5 Contribution"]
  N6["Power and sample"]
  N7["Approvals"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX015"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["3.1 Hypotheses"]
  N2["3.2 Design"]
  N3["3.3 Rings"]
  N4["3.4 Threats to validity"]
  N5["3.5 Contribution"]
  N6["Power and sample"]
  N7["Approvals"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX015"]

[Learning outcomes]
    |
    v
[3.1 Hypotheses]
    |
    v
[3.2 Design]
    |
    v
[3.3 Rings]
    |
    v
[3.4 Threats to validity]
    |
    v
[3.5 Contribution]
    |
    v
[Power and sample]
    |
    v
[Approvals]
    |
    v
[Worked example] --> [Exercise EX015]

Learning outcomes

You will (1) specify hypotheses, design and outcomes; (2) justify a quasi-experimental design; (3) plan threats to validity.

3.1 Hypotheses

H1: ring-based deployment reduces the 95th-percentile days to patch. H2: ring-based deployment reduces the update failure rate. Both are directional and pre-registered. The 95th percentile was chosen because the slowest endpoints, not the average, decide how long a vulnerability stays exposed across a site. Directional hypotheses commit Yusuf to reporting a result in the opposite direction as evidence against them, not as an unexpected finding to explain away. Both measures can be computed from data the trust already collects. Each hypothesis has one test fixed in the registered plan, so Yusuf cannot choose a different test after seeing the trust's data.

3.2 Design

Harrowgate has two comparable hospital sites. With the trust's agreement, one site adopts rings for six months while the other continues flat deployment. Random assignment of individual endpoints is impossible because wards share devices and staff, so the design is a site-level quasi-experiment. The choice of site was agreed with the trust before the baseline period, based on which site's desktop team could support the change. Yusuf records that choice and its reason, because a site chosen for its readiness might differ in ways that also affect outcomes.

3.3 Rings

Ring 0 is 40 IT devices, patched first; ring 1 is 160 devices in pilot wards; ring 2 is the remaining 800. Each ring waits for the previous ring's failure signals before proceeding, following the staged approach discussed in SP 800-40r4. Ring 0 contains IT staff devices because their users can report problems precisely and tolerate disruption. Ring 1's pilot wards were chosen with the clinical leads to include common device types. If ring 0 or ring 1 shows failures, deployment to the next ring pauses until the cause is known.

3.4 Threats to validity

Sites may differ in device age or staff habits. Yusuf collects baseline data from both sites for three months before the change, so differences can be measured. Seasonal workload affects both sites equally. The three-month baseline lets him see whether the sites already differed before rings began, and by how much. He also records device age and operating-system mix at each site, so any difference in failures can be compared with those factors. The analysis reports the effect both raw and after subtracting baseline.

3.5 Contribution

The thesis contributes measured effects of a common but untested practice, a codebook for failure tickets and an open, synthetic-structure dataset with analysis code. Each contribution can be checked separately: the effect estimates against the published code and data, the codebook against its agreement figures, and the dataset against its digests. A practitioner can use the codebook alone even if the effect does not generalise to their organisation. The thesis states the boundary of the effect in the same paragraph as the claim.

Power and sample

With 1,000 endpoints per site and monthly cycles, Yusuf estimates that six months gives enough failures to detect a difference of about one percentage point in failure rate, based on the trust's historical rate of roughly three per cent. He records this calculation in the pre-registration, along with the decision not to extend the study if results are unclear, which prevents stopping when results look favourable.

Approvals

The proposal is approved by Calder's research ethics committee and the trust's information governance lead. Both approvals specify that no clinical system will be changed solely for research; rings use the trust's existing deployment tool.

Worked example

Prof. Solberg asks why not randomise wards. Yusuf explains that ward devices are managed by shared policies at site level, so splitting a site would mix treatments; site-level assignment with baseline comparison is the honest option.

Synthetic case. Synthetic.

Errors and limits

Two sites give limited generalisation; the thesis must say so.

Exercises

Synthesis

Propose a design that is honest about what assignment is possible.

Continue studying — chapter 03 of 08

← Chapter 02 (draft) · Course chapter index · Chapter 04 (draft) →

Worked answers

EX015. H1: rings reduce 95th-percentile days to patch; H2: rings reduce failure rate.

EX016. Shared devices and policies prevent individual randomisation.

EX017. Ring 0: 40, ring 1: 160, ring 2: 800.

EX018. To measure pre-existing site differences.

EX019. Measured effects, a codebook and an open dataset with code.

EX020. Site-level policies would mix treatments.