Chapter 6 — Identity and access management: controlled evaluation
Lesson map
Lesson 06 published. Map, example, EX026.
flowchart TD
N0["Learning outcomes"]
N1["6.1 Pilot design"]
N2["6.2 Results"]
N3["Lab — Out-of-window activations"]
N4["6.3 Reviewing activations"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX026"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["6.1 Pilot design"]
N2["6.2 Results"]
N3["Lab — Out-of-window activations"]
N4["6.3 Reviewing activations"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX026"]
[Learning outcomes]
|
v
[6.1 Pilot design]
|
v
[6.2 Results]
|
v
[Lab — Out-of-window activations]
|
v
[6.3 Reviewing activations]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX026]
Learning outcomes
You will (1) pilot the chosen design with a controlled comparison; (2) measure outcomes against a baseline; (3) review out-of-window activations.
6.1 Pilot design
Corran approves Design B. Marta pilots it for six weeks with four of the eight administrators, chosen to cover day and night shifts, while the other four continue as before. Both groups' activity is measured over the same weeks. The comparison is not randomised, since shift patterns made that impractical, so Marta records differences between the groups, such as workload, that could affect results. Measures include privilege-hours, elevation requests, time to obtain elevation and any operational delays reported by warehouse managers.
6.2 Results
The pilot group's privilege-hours fall from 672 a week to about 9, while the comparison group stays at 672. Median time to obtain elevation is under two minutes, because the built-in workflow approves standard requests automatically during change windows. Warehouse managers report one delay, when an administrator's security key was left at home; a second key, registered and kept in the office safe, solves this. These results suggest Design B reduces exposure greatly with little disruption, though a six-week pilot cannot show long-term effects.
Lab — Out-of-window activations
"""Flag administrator activations outside approved change windows in synthetic logs (stdlib only)."""
acts = [("adm01", "Tue", 19), ("adm02", "Wed", 10), ("adm05", "Sat", 2), ("adm02", "Thu", 22), ("adm06", "Mon", 9),
("adm03", "Sun", 14), ("adm01", "Fri", 11), ("adm04", "Tue", 20), ("adm02", "Fri", 16), ("adm05", "Wed", 13)]
def in_window(day, h): return (day in ("Mon", "Tue", "Wed", "Thu", "Fri") and 8 <= h < 18) or (day == "Tue" and 18 <= h < 22)
out = [a for a in acts if not in_window(a[1], a[2])]
print(f"activations={len(acts)} outside_window={len(out)}")
for a in out: print(" review:", a)
Run with python3 lab_window.py. Verified output:
activations=10 outside_window=3
review: ('adm05', 'Sat', 2)
review: ('adm02', 'Thu', 22)
review: ('adm03', 'Sun', 14)
6.3 Reviewing activations
Of ten synthetic activations, three fall outside approved windows: adm05 at 02:00 on Saturday, adm02 at 22:00 on Thursday and adm03 on Sunday afternoon. Each is reviewed with the administrator. Two are genuine emergencies with incident tickets; one is a routine task done at a convenient time, which is not wrong but should be scheduled. Marta reports this as evidence that monitoring works and that most out-of-window use has legitimate reasons, which reassures administrators that monitoring is about understanding, not catching people out.
Worked example
The pilot cuts privilege-hours from 672 to about 9 a week with two-minute elevation; three of ten activations fall outside windows, two of them documented emergencies.
Synthetic case. Groups, hours and activations are synthetic.
Errors and limits
Treating every out-of-window activation as misconduct ignores genuine emergencies and damages trust.
Exercises
- EX026 — Pilot comparison. How is the pilot comparison structured? Rubric: 3 = four admins on Design B, four unchanged, same weeks;; 2 = two groups;; 1 = vague;; 0 = none.
- EX027 — Elevation time. What is the median time to obtain elevation? Rubric: 3 = under two minutes;; 2 = minutes;; 1 = hours;; 0 = none.
- EX028 — Key fix. How was the forgotten key problem solved? Rubric: 3 = second key kept in the office safe;; 2 = spare key;; 1 = password;; 0 = none.
- EX029 — Off-window count. How many activations fall outside windows in the lab? Rubric: 3 = three of ten;; 2 = three;; 1 = vague;; 0 = none.
- EX030 — Off-window outcomes. What did review find about the out-of-window activations? Rubric: 3 = two emergencies, one routine task to schedule;; 2 = mostly legitimate;; 1 = misconduct;; 0 = none.
Synthesis
A controlled pilot with a comparison group and reviewed activations shows both the benefit and the practical effects of the design.
Continue studying — chapter 06 of 12
← Chapter 05 (draft) · Course chapter index · Chapter 07 (draft) →