◇ OpenCyber Academy
Master's · Chapter 4 · English

Chapter 4 — Human factors and cyber policy: competing explanations

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 4 — Human factors and cyber policy: competing explanations

Lesson map

Lesson 04 published. Map, example, EX021.

flowchart TD
  N0["Learning outcomes"]
  N1["4.1 Possible explanations"]
  N2["4.2 Checks for each"]
  N3["Lab — Interrupted time series of monthly resets per"]
  N4["4.3 What the checks show"]
  N5["4.4 Deciding what is supported"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX021"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["4.1 Possible explanations"]
  N2["4.2 Checks for each"]
  N3["Lab — Interrupted time series of monthly resets per"]
  N4["4.3 What the checks show"]
  N5["4.4 Deciding what is supported"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX021"]

[Learning outcomes]
    |
    v
[4.1 Possible explanations]
    |
    v
[4.2 Checks for each]
    |
    v
[Lab — Interrupted time series of monthly resets per]
    |
    v
[4.3 What the checks show]
    |
    v
[4.4 Deciding what is supported]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX021]

Learning outcomes

You will (1) list explanations for a change in reset counts; (2) design checks for each; (3) decide what the evidence supports.

4.1 Possible explanations

Early data shows password resets fell after the policy change. Dr Nakamura asks Sofia to list every plausible explanation before claiming success. Resets could fall because forced changes stopped; because the password manager helped people remember; because a new self-service reset tool moved requests away from the help desk; because staffing fell over summer; or because the help desk changed how it logged tickets. Listing explanations prevents her from accepting the first story that fits her hopes. She writes each explanation on a separate line of a table so that she can attach a specific check and its result to every one.

4.2 Checks for each

Sofia designs a check for each. For self-service, she obtains self-service reset counts and adds them to help desk counts. For staffing, she divides resets by the number of active accounts. For logging changes, she reads the help desk change log and interviews the help desk lead. For seasonal effects, she compares the same months in the previous year. For the password manager, she compares departments with high and low manager adoption. Each check targets one explanation directly rather than relying on a general impression.

Lab — Interrupted time series of monthly resets per 1,000 accounts

# Interrupted time series on resets per 1,000 accounts: 18 months before, 12 after (synthetic).
import random
random.seed(3)
y = [62 - 0.2 * t + random.gauss(0, 2) for t in range(18)] + [39 - 0.25 * (t - 18) + random.gauss(0, 2) for t in range(18, 30)]
def fit(xs, ys):
    n = len(xs); mx = sum(xs) / n; my = sum(ys) / n
    b = sum((x - mx) * (v - my) for x, v in zip(xs, ys)) / sum((x - mx) ** 2 for x in xs)
    return my - b * mx, b
a0, b0 = fit(range(18), y[:18]); a1, b1 = fit(range(18, 30), y[18:])
pred = a0 + b0 * 18; obs = a1 + b1 * 18
print(f"before: level at month 18 predicted {pred:.1f}, slope {b0:+.2f}/month")
print(f"after: level at month 18 fitted {obs:.1f}, slope {b1:+.2f}/month")
print(f"level change {obs - pred:+.1f} per 1,000 accounts; slope change {b1 - b0:+.2f}")

Run with python3 lab_its.py. Verified output:

before: level at month 18 predicted 56.9, slope -0.37/month
after: level at month 18 fitted 38.1, slope -0.27/month
level change -18.8 per 1,000 accounts; slope change +0.09

4.3 What the checks show

Self-service resets account for part of the drop but not all: total resets per thousand accounts still fall after the change. Staffing changes are small. The help desk logging change happened four months later, so it cannot explain the immediate drop. The lab estimates a fall in level at the change point with little change in trend. Departments with higher manager adoption show slightly larger falls, which is consistent with but does not prove a manager effect, since adopters may differ in other ways.

4.4 Deciding what is supported

Sofia writes her conclusion in levels of confidence. Strongly supported: total resets per thousand accounts fell at the change point. Moderately supported: ending forced changes contributed, since the timing matches and alternatives are mostly ruled out. Weakly supported: the password manager contributed. Not tested: whether security improved. Dr Nakamura praises the structure, noting that ranking explanations by evidence is more honest and more useful to policy makers than a single cause stated with false certainty. She presents the four levels as a short table in the discussion chapter so that readers can see at a glance how strongly each claim is supported.

Worked example

Sofia lists five explanations for falling resets, designs specific checks, runs an interrupted time series and ranks conclusions by confidence.

Synthetic case. Data and departments are synthetic.

Errors and limits

Department comparisons are confounded by adoption choices. Time series need enough points before and after.

Exercises

Synthesis

Listing and checking competing explanations lets a study rank its conclusions honestly by strength of evidence.

Continue studying — chapter 04 of 08

← Chapter 03 (draft) · Course chapter index · Chapter 05 (draft) →