Chapter 3 — Original contribution proposal
Lesson map
Lesson 03 published. Map, example, EX015.
flowchart TD
N0["Learning outcomes"]
N1["3.1 Hypotheses"]
N2["3.2 Measures"]
N3["3.3 Design and comparison"]
N4["3.4 Threats to validity"]
N5["3.5 Ethics of the intervention"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX015"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["3.1 Hypotheses"]
N2["3.2 Measures"]
N3["3.3 Design and comparison"]
N4["3.4 Threats to validity"]
N5["3.5 Ethics of the intervention"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX015"]
[Learning outcomes]
|
v
[3.1 Hypotheses]
|
v
[3.2 Measures]
|
v
[3.3 Design and comparison]
|
v
[3.4 Threats to validity]
|
v
[3.5 Ethics of the intervention]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX015]
Learning outcomes
You will (1) state hypotheses and their tests; (2) design the comparison of review procedures; (3) plan for threats to validity.
3.1 Hypotheses
Rhea registers two hypotheses with her supervisor before seeing partner data. H1: across business units, higher role-model complexity is associated with a higher rate of excess privilege, tested with a rank correlation because neither measure is expected to be normally distributed. H2: units using the structured review remove a greater share of coded excess entitlements than units using the usual review, tested by comparing removal rates. For each, she states what result would count against it, such as a correlation near zero or no difference in removal. Writing the falsifying result in advance prevents later reinterpretation of whatever the data shows.
3.2 Measures
The main outcome is the excess rate: entitlements coded excess, dormant or toxic, divided by all entitlements in a unit. Removal rate is the share of those entitlements removed after a review cycle. Complexity is measured by mean roles per user and mean entitlements per role, combined into a ranked index. Rhea also records reviewer time, because a procedure that removes more excess but takes far longer may not be adopted. Every measure has a written definition, a computation script tested on synthetic data and a note on its limitations, so examiners can check the numbers independently.
3.3 Design and comparison
Business units are randomly assigned, within each partner, to the structured review or the usual review for one cycle. Randomising within partners balances differences between organisations, such as directory products or culture. The structured review shows each reviewer the entitlement's plain-language description, its last use and the role it came from, and limits a sitting to fifty decisions. The usual review is left exactly as each partner runs it. Rhea cannot blind reviewers to their condition, but she blinds the coders who later judge whether removals were correct, which protects the main outcome from her own expectations.
3.4 Threats to validity
Rhea lists threats with mitigations. Contamination, where reviewers in one condition learn the other's method, is reduced by assigning whole units and asking partners not to share materials during the cycle. Seasonal effects, such as year-end staff changes, are handled by running both conditions in the same period. Measurement error in usage logs, which may miss some access paths, is checked by comparing logs with administrator knowledge on a sample. Small numbers of units limit statistical power, so Rhea reports effect sizes with intervals rather than relying on significance alone. Each remaining threat is stated openly in the thesis.
3.5 Ethics of the intervention
Changing a review process affects real people: wrongly removing access can stop a nurse reaching patient records during a shift. Rhea's protocol therefore requires that all removals follow each partner's normal change procedure, that any removal can be reversed within one working day, and that partners can stop the study at any time. Reviewers are told the purpose of the study and agree to take part. The ethics committee reviews these safeguards, and Rhea records every reversal as data, because wrongful removals are a cost of the procedure that must be reported alongside its benefits.
Worked example
Rhea's proposal states two falsifiable hypotheses, defines four measures, randomises units within partners, blinds outcome coders and lists threats with mitigations and ethical safeguards.
Synthetic case. Hypotheses, units and partners are synthetic.
Errors and limits
Without randomisation and blinded outcome coding, any difference in removals could reflect which units were chosen or what the researcher hoped to see.
Exercises
- EX015 — H1 test. Which test does H1 use and why? Rubric: 3 = rank correlation, measures not expected normal;; 2 = rank correlation;; 1 = t-test;; 0 = none.
- EX016 — Excess rate. Define the excess rate. Rubric: 3 = excess, dormant or toxic entitlements over all entitlements in a unit;; 2 = share of excess;; 1 = count;; 0 = none.
- EX017 — Randomisation level. At what level are units randomised, and why? Rubric: 3 = within partners, to balance organisational differences;; 2 = within partners;; 1 = vague;; 0 = none.
- EX018 — Blinding target. Who is blinded in the design? Rubric: 3 = outcome coders judging removals;; 2 = coders;; 1 = reviewers;; 0 = none.
- EX019 — Reviewer time reason. Why record reviewer time? Rubric: 3 = a slower procedure may not be adopted;; 2 = adoption;; 1 = no reason;; 0 = none.
- EX020 — Reversal safeguard. What safeguard protects staff from wrongful removal? Rubric: 3 = removals reversible within one working day via normal change;; 2 = reversible;; 1 = vague;; 0 = none.
Synthesis
A strong proposal fixes hypotheses, measures and design before data, and plans honestly for threats and harms.
Continue studying — chapter 03 of 08
← Chapter 02 (draft) · Course chapter index · Chapter 04 (draft) →