◇ OpenCyber Academy
Master's · Chapter 6 · English

Chapter 6 — Data collection and analysis

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 6 — Data collection and analysis

Lesson map

Lesson 06 published. Map, example, EX033.

flowchart TD
  N0["Learning outcomes"]
  N1["6.1 Collecting data"]
  N2["6.2 Analysing survey change"]
  N3["Lab — Bootstrap interval for change in writing passw"]
  N4["6.3 Coding interviews"]
  N5["6.4 Bringing results together"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX033"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["6.1 Collecting data"]
  N2["6.2 Analysing survey change"]
  N3["Lab — Bootstrap interval for change in writing passw"]
  N4["6.3 Coding interviews"]
  N5["6.4 Bringing results together"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX033"]

[Learning outcomes]
    |
    v
[6.1 Collecting data]
    |
    v
[6.2 Analysing survey change]
    |
    v
[Lab — Bootstrap interval for change in writing passw]
    |
    v
[6.3 Coding interviews]
    |
    v
[6.4 Bringing results together]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX033]

Learning outcomes

You will (1) collect survey and interview data ethically; (2) analyse change in survey responses; (3) code interviews into themes.

6.1 Collecting data

Sofia runs the survey three months after the policy change, using the same questions the trust had asked in an earlier staff survey before the change. The before survey was anonymous and had the same wording for two questions on reuse and writing passwords down, which lets her compare. Four hundred and twelve staff respond after the change, against three hundred and ninety before. She conducts twelve interviews across nursing, medical, administrative and estates staff, stopping when new interviews add few new ideas. All interviews are recorded with consent.

6.2 Analysing survey change

Sofia compares the share of staff reporting that they write passwords down. The lab estimates the before and after shares and a bootstrap interval for the difference, which resamples the responses many times to show how much the difference could vary by chance. The share fell by a few percentage points, and the interval does not cross zero, so the fall is unlikely to be chance alone. She notes the samples are different people, since the surveys were anonymous, so this is a comparison of groups, not of individuals.

Lab — Bootstrap interval for change in writing passwords down

# Bootstrap interval for the change in share reporting writing passwords down (synthetic anonymous surveys).
import random
rng = random.Random(9)
before = [1] * 82 + [0] * 308    # 390 respondents
after = [1] * 62 + [0] * 350     # 412 respondents
pb, pa = sum(before) / len(before), sum(after) / len(after)
diffs = []
for _ in range(4000):
    b = [rng.choice(before) for _ in before]; a = [rng.choice(after) for _ in after]
    diffs.append(sum(a) / len(a) - sum(b) / len(b))
diffs.sort()
print(f"before {pb:.1%}, after {pa:.1%}, difference {pa - pb:+.1%}")
print(f"95% bootstrap interval {diffs[100]:+.1%} to {diffs[3899]:+.1%}")

Run with python3 lab_boot.py. Verified output:

before 21.0%, after 15.0%, difference -6.0%
95% bootstrap interval -11.3% to -0.8%

6.3 Coding interviews

Sofia codes interview transcripts using a frame built from her theories and refined by reading. Themes include time pressure on shift, workarounds such as shared notes, relief at fewer changes and confusion about the password manager. A second coder codes three transcripts independently; they agree on most codes and discuss differences. The interviews explain the numbers: staff describe forced changes as interrupting patient care, and several say the password manager was introduced without training, limiting adoption. Agreed codes are recorded in a shared codebook with definitions and an example quote for each, so that the frame can be reused by others.

6.4 Bringing results together

Sofia combines findings. Resets fell at the change point; fewer staff report writing passwords down; interviews describe relief and less disruption. Manager adoption is lower than hoped, explained by missing training. She judges that the evidence supports the compliance budget prediction about reduced burden, while planned behaviour helps explain low manager adoption through perceived difficulty. Integrating methods shows a fuller picture than any one method, and also where they disagree, such as staff overestimating how often they still reset. She shows these links in a joint display table.

Worked example

Sofia compares anonymous before and after surveys with a bootstrap interval, codes twelve interviews with a second coder and integrates findings with theory.

Synthetic case. Surveys, interviews and figures are synthetic.

Errors and limits

Anonymous repeated surveys compare groups, not individuals. Saturation judgements are subjective.

Exercises

Synthesis

Combining counts, survey change and interview themes gives a fuller and more honest picture than any single method.

Continue studying — chapter 06 of 08

← Chapter 05 (draft) · Course chapter index · Chapter 07 (draft) →