Chapter 29 — Information security: measurement
Lesson map
Lesson 29 published. Map, example, EX085.
flowchart TD
N0["Learning outcomes"]
N1["29.1 Measuring password guessing"]
N2["29.2 The login lab"]
N3["Lab — Detecting password-guessing bursts in a synthe"]
N4["29.3 Why rate beats total"]
N5["29.4 Reporting"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX085"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["29.1 Measuring password guessing"]
N2["29.2 The login lab"]
N3["Lab — Detecting password-guessing bursts in a synthe"]
N4["29.3 Why rate beats total"]
N5["29.4 Reporting"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX085"]
[Learning outcomes]
|
v
[29.1 Measuring password guessing]
|
v
[29.2 The login lab]
|
v
[Lab — Detecting password-guessing bursts in a synthe]
|
v
[29.3 Why rate beats total]
|
v
[29.4 Reporting]
|
v
[Worked example] --> [Exercise EX085]
Learning outcomes
You will (1) design a detection measure; (2) distinguish attack patterns from benign noise; (3) report detection results honestly.
29.1 Measuring password guessing
Saltmarsh wants to know how often accounts face password guessing. Lindiwe defines a measure: an account is flagged if it has ten or more failed logins within any five-minute window. Ordinary mistyping produces a few failures spread across a day, while guessing produces many in quick succession. She chooses the window and threshold before looking at data, and records them so later results can be compared with the same definition. The security lead approves the definition before Lindiwe runs it on any data.
29.2 The login lab
The lab builds a synthetic day of 1,672 failed logins across 863 accounts. Three accounts are flagged: u17, u402 and u977, with 40, 42 and 40 failures respectively, all in short bursts. The account with the most failures overall, u1200, has 61, but they are spread every few minutes across the day, so it is not flagged. Lindiwe investigates and finds a broken synchronisation client retrying with an old password. She records the explanation beside the account in her report, so nobody mistakes the client's failures for an attack. The service desk confirms the user had changed her password the previous week.
Lab — Detecting password-guessing bursts in a synthetic authentication log
"""Detect password-guessing bursts in a synthetic authentication log (stdlib only)."""
import random
from collections import defaultdict
random.seed(5)
events = []
for u in range(1500): # ordinary mistyping
for _ in range(random.choice([0, 0, 0, 1, 1, 2, 3])):
events.append((f"u{u}", random.uniform(0, 86400)))
for u in ["u17", "u402", "u977"]: # guessing bursts
start = random.uniform(0, 80000)
events += [(u, start + random.uniform(0, 240)) for _ in range(40)]
for u in ["u1200"]: # a broken sync client retrying slowly
events += [(u, k * 400.0) for k in range(60)]
by = defaultdict(list)
for u, t in events: by[u].append(t)
flag = []
for u, ts in by.items():
ts.sort()
if any(ts[j + 9] - ts[j] <= 300 for j in range(len(ts) - 9)):
flag.append(u)
total = sorted(by, key=lambda u: -len(by[u]))[:4]
print(f"failed logins {len(events)}; accounts with failures {len(by)}")
print(f"flagged (10 failures within 5 min): {sorted(flag)}")
print(f"top accounts by total failures: {[(u, len(by[u])) for u in total]}")
Run with python3 lab_logins.py. Verified output:
failed logins 1672; accounts with failures 863
flagged (10 failures within 5 min): ['u17', 'u402', 'u977']
top accounts by total failures: [('u1200', 61), ('u402', 42), ('u17', 40), ('u977', 40)]
29.3 Why rate beats total
Ranking accounts by total failures would have put the broken client first and mixed it with real guessing. The window measure separates fast guessing from slow retries. Lindiwe notes that attackers can also guess slowly to avoid detection, so the measure catches noisy attacks but not patient ones. She adds a second, weaker indicator: accounts with failures from many different addresses in a day. Prof. Hale appreciates that she states what the measure cannot see. She documents both indicators together in one short note.
29.4 Reporting
Lindiwe reports the three flagged accounts to the security team, which resets their passwords and confirms multi-factor authentication is enabled. She also reports the broken client to the service desk, which fixes it. Her monthly report shows the number of flagged accounts, the definition used and its known blind spot. She learns that a measure is useful only when its definition and limits travel with the number. The security lead circulates the report to the IT managers each month, and the definition is added to the security metrics handbook.
Worked example
Lindiwe flags three accounts with bursts of 40 to 42 failures among 1,672 failed logins, separates a slow-retrying client with 61 failures and reports the measure with its blind spot.
Synthetic case. Logs are synthetic; outputs come from lab_logins.py.
Errors and limits
Ranking by total failures confuses broken clients with attacks.
Exercises
- EX085 — Flagged accounts. Which accounts are flagged? Rubric: 3 = u17, u402 and u977;; 2 = partly correct;; 1 = wrong;; 0 = none.
- EX086 — Top account. Why is u1200 not flagged? Rubric: 3 = failures spread across the day;; 2 = partly correct;; 1 = vague;; 0 = none.
- EX087 — Blind spot. What does the window measure miss? Rubric: 3 = slow, patient guessing;; 2 = partly correct;; 1 = vague;; 0 = none.
Continue studying — lesson 29 (chapter 29 of 50)
← Chapter 28 · Course chapter index · Chapter 30 →
Continue to the next authored lesson.