OpenCyber AcademyLearning outcomes
You will (1) frame a doctoral question about excess privilege in identity systems; (2) design a valid coding scheme for entitlements; (3) establish ethical and data-handling rules for partner identity data.
1.1 The case
Rhea Mwangi is a doctoral candidate at the fictional Thornfield University, supervised by Prof. Daniel Osei. Her partners are three fictional public bodies, led by Calder Valley Health Trust, whose directories have grown over a decade to thousands of accounts and hundreds of roles. Managers approve access reviews every quarter, yet auditors keep finding staff with access they no longer need. Rhea's thesis asks two questions: does role-model complexity predict the rate of excess privilege, and does a structured access recertification procedure remove more excess than the usual approve-all review? Saltzer and Schroeder (1975) named least privilege as a design principle; her work asks how well organisations actually achieve it.
1.2 Defining excess
A study of excess privilege needs a definition two people would apply identically. Rhea codes each entitlement, meaning one permission or group membership held by one account, into four categories: needed, with a documented business purpose and recent use; excess, held without a purpose the role requires; dormant, not used for ninety days and with no documented seasonal need; and toxic, a combination that breaks a separation-of-duties rule, such as raising and approving the same payment. Each category has written decision rules and worked synthetic examples, drawing on the access control family of NIST SP 800-53 Rev. 5. The codebook is fixed before any partner data is coded.
Lab — Coder agreement
"""Cohen's kappa for two coders labelling synthetic entitlements (stdlib only)."""
from collections import Counter
cats = ["needed", "excess", "dormant", "toxic"]
a = ["needed"]*20 + ["excess"]*9 + ["dormant"]*7 + ["toxic"]*4
b = list(a)
for i, v in [(3, "excess"), (12, "dormant"), (22, "needed"), (31, "excess")]: b[i] = v
n = len(a); po = sum(x == y for x, y in zip(a, b)) / n
ca, cb = Counter(a), Counter(b); pe = sum(ca[c]*cb[c] for c in cats) / n**2
print(f"items={n} agree={sum(x == y for x, y in zip(a, b))} po={po:.2f} pe={pe:.2f} kappa={(po-pe)/(1-pe):.2f}")
for i, (x, y) in enumerate(zip(a, b)):
if x != y: print(f" item {i:02d}: coder A={x:8} coder B={y}")
Run with python3 lab_kappa.py. Verified output:
items=40 agree=36 po=0.90 pe=0.33 kappa=0.85
item 03: coder A=needed coder B=excess
item 12: coder A=needed coder B=dormant
item 22: coder A=excess coder B=needed
item 31: coder A=dormant coder B=excess
1.3 Reading agreement
Two trained coders independently label forty synthetic entitlements and agree on thirty-six, an observed agreement of 0.90. Cohen's kappa, which removes the agreement expected by chance from the category frequencies, is 0.85. Landis and Koch (1977) would describe this as almost perfect, though Rhea treats their labels as rough guidance rather than thresholds. The four disagreements matter more than the summary: two separate needed from excess or dormant, where coders read the role description differently, and two separate excess from dormant. Each produces a sharper decision rule, such as checking the role catalogue before the usage log, and the exercise is repeated on fresh items after the change.
1.4 Integrity and data handling
Identity data is sensitive: it names staff, reveals who can reach patient records and shows where controls are weak. Rhea's data agreement lets partners replace names and account identifiers with consistent pseudonyms before export, keeps files on an encrypted university server with access logged, and forbids publishing any entitlement name that would identify a system. All examples in her thesis and in this book are synthetic. Ethics approval covers interviews with reviewers and administrators, who can withdraw at any time. She never logs in to partner systems or tests their controls; every observation comes from exports the partners choose to share, under their own change procedures.
1.5 Validity from the start
Rhea separates three validity questions early. Construct validity asks whether her four categories really capture excess privilege as practitioners understand it, so she checks the codebook with two partner administrators. Internal validity asks whether any observed change can be attributed to the review procedure rather than to staff turnover or a system migration, which shapes her design. External validity asks whether results from three public bodies generalise, which she will answer modestly. Writing these down at the start keeps later decisions honest, because each design choice can be checked against the threat it is meant to address.
Worked example
Rhea writes the research questions, the four-category codebook and the data agreement, then runs the forty-item agreement exercise. Kappa of 0.85 and two clear disagreement patterns lead to two codebook clarifications before real coding begins.
Synthetic case. Rhea Mwangi, Thornfield University, Calder Valley Health Trust and all figures and entitlements are synthetic.
Errors and limits
Kappa summarises agreement but hides where disagreement lies; always inspect disagreements, and remember that high agreement does not prove the categories are the right ones.
Exercises
- EX001 — Entitlement unit. In Rhea's study, what is one entitlement?
Show marking criteria
Rubric: 3 = one permission or group membership held by one account;; 2 = one permission;; 1 = one account;; 0 = none. - EX002 — Toxic category. Which category covers raising and approving the same payment?
Show marking criteria
Rubric: 3 = toxic, a separation-of-duties breach;; 2 = toxic;; 1 = excess;; 0 = none. - EX003 — Kappa value. State the observed agreement and kappa from the lab.
Show marking criteria
Rubric: 3 = 0.90 and 0.85 with meaning;; 2 = both numbers;; 1 = one;; 0 = none. - EX004 — Chance correction. What does kappa add beyond raw agreement?
Show marking criteria
Rubric: 3 = removes agreement expected by chance from category frequencies;; 2 = chance;; 1 = vague;; 0 = none. - EX005 — Disagreement use. How should the four disagreements be used?
Show marking criteria
Rubric: 3 = sharpen decision rules and repeat on fresh items;; 2 = refine codebook;; 1 = ignore;; 0 = none. - EX006 — Data boundary. Name two protections in Rhea's data agreement.
Show marking criteria
Rubric: 3 = pseudonyms, encrypted logged storage, no publication of identifying names;; 2 = two;; 1 = one;; 0 = none. - EX007 — Internal validity threat. Give one threat to attributing change to the review procedure.
Show marking criteria
Rubric: 3 = staff turnover or system migration;; 2 = one threat;; 1 = vague;; 0 = none.
Synthesis
A defensible doctoral study of excess privilege starts with a precise codebook, measured coder agreement and strict handling of sensitive identity data.