Chapter 30 — Laboratory II
Lesson map
Lesson 30 published. Map, example, EX088.
flowchart TD
N0["Learning outcomes"]
N1["30.1 Purpose"]
N2["30.2 Setup"]
N3["Lab — correlation against known cases"]
N4["30.3 Reading the result"]
N5["30.4 Analysing the miss"]
N6["30.5 Adjusting"]
N7["30.6 Re-test"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX088"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["30.1 Purpose"]
N2["30.2 Setup"]
N3["Lab — correlation against known cases"]
N4["30.3 Reading the result"]
N5["30.4 Analysing the miss"]
N6["30.5 Adjusting"]
N7["30.6 Re-test"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX088"]
[Learning outcomes]
|
v
[30.1 Purpose]
|
v
[30.2 Setup]
|
v
[Lab — correlation against known cases]
|
v
[30.3 Reading the result]
|
v
[30.4 Analysing the miss]
|
v
[30.5 Adjusting]
|
v
[30.6 Re-test]
|
v
[Worked example] --> [Exercise EX088]
Learning outcomes
You will (1) test a correlation detection against known cases; (2) analyse why a known case was missed; (3) adjust and re-test the rule; (4) document limits after adjustment.
30.1 Purpose
Laboratory II tests the correlation rule from chapter 26 against the synthetic reconstruction of the campaign, where the right answers are known.
Known answers make the test meaningful: s2047 and s3310 were both compromised, and s1001 was not. A correct rule must alert on both and stay quiet on s1001. Testing on these cases before deploying shows whether the rule would have helped during the campaign itself. A rule that fails on these known cases would certainly fail on new ones. Passing them is only a first step, since real attackers will vary their timing, which is why the lab also asks what the rule would miss.
30.2 Setup
The lab contains synthetic sign-ins and mailbox-rule events for s2047, s3310 and s1001, with times in minutes since midnight.
The times match the campaign: minute 134 is 02:14, when s2047 signed in, and minute 151 is 02:31, s3310's successful sign-in. Using minutes since midnight keeps the lab simple, though the real rule uses full timestamps in UTC, as the time-handling chapters require. The data contain only these three accounts and four events, which is small enough to check every result by hand. Lucas computes the expected output on paper before running the script, so any difference points to a mistake in either the rule or his understanding of it.
Lab — correlation against known cases
"""Correlate risky sign-ins with mailbox-rule creation within 60 minutes (synthetic)."""
signins = [("s2047", 134, "NL", "new-country"), ("s3310", 151, "NL", "new-country"), ("s1001", 545, "GB", "normal")]
rules = [("s2047", 140, "forward to external"), ("s1001", 600, "move newsletters"), ("s3310", 300, "delete if 'phishing'")]
for user, t_sign, country, risk in signins:
if risk == "normal": continue
for u, t_rule, desc in rules:
if u == user and 0 <= t_rule - t_sign <= 60:
print(f"ALERT {user}: risky sign-in {country} at minute {t_sign} then rule '{desc}' at minute {t_rule}")
elif u == user:
print(f"no match {user}: rule '{desc}' {t_rule - t_sign} min after sign-in (outside window)")
Run with python3 lab_correlate.py. Verified output:
ALERT s2047: risky sign-in NL at minute 134 then rule 'forward to external' at minute 140
no match s3310: rule 'delete if 'phishing'' 149 min after sign-in (outside window)
30.3 Reading the result
The rule alerts on s2047: a risky sign-in at minute 134 (02:14) followed by an external forwarding rule six minutes later. It misses s3310: the deletion rule came 149 minutes after the risky sign-in, outside the 60-minute window.
The s1001 events produce no output because its sign-in was normal and the loop skips it. That silence is correct: the newsletter rule should not alert. The lab therefore shows one true positive, one false negative and one true negative. A false negative on a real compromise is the most important finding.
30.4 Analysing the miss
s3310's attacker waited over two hours before hiding warnings, perhaps after seeing replies. Attackers do not follow the defender's assumptions about timing.
Lucas's original window assumed attackers act fast, which s2047's did. s3310's attacker created the deletion rule at minute 300, about 05:00, perhaps after seeing replies questioning the email. The gap shows that a rule built from one case risks matching only that case. Lucas also asked what other timings attackers might use. An attacker who waits a day before acting would escape even a six-hour window, which is one reason the separate keyword rule does not depend on timing at all.
30.5 Adjusting
Extending the window to 6 hours catches s3310 but risks linking unrelated events. Lucas adds a second, separate rule: any keyword-deletion rule matching words like "phishing" or "hacked" alerts on its own, regardless of sign-in, because such rules are rare and suspicious in themselves.
The two changes work differently. The longer window still depends on a risky sign-in, so it catches s3310 only because its sign-in was flagged. The keyword rule needs no sign-in at all, so it would also catch an attacker whose sign-in looked normal. Together they cover both timing and the absence of a sign-in signal.
30.6 Re-test
With the 6-hour window plus the keyword rule, both compromised accounts alert and s1001's ordinary folder rule does not.
Rerunning the same three accounts confirms the expected outcomes, but the dataset is tiny. Lucas notes that a longer window could match more ordinary rules in real traffic, which is why the monitoring described below follows. A pass on three synthetic accounts is necessary but not sufficient. The next real test is a month of live traffic. Lucas sets a threshold for acceptable false alerts before that month starts, so he cannot decide afterwards that whatever happened was acceptable.
Monitoring after change
For a month after the change, Lucas reviews every match of both rules each Friday and records whether it was correct. If false alerts exceed two per week, he will revisit the window.
Worked example
Lucas's lab report: "60-minute correlation missed s3310 (rule after 149 minutes). Changes: window 6 hours; new stand-alone alert for deletion rules containing warning keywords. Re-test: 2 of 2 compromised accounts detected, 0 false alerts on test data. Limit: test data is small; monitor false alerts for one month."
Synthetic case. Events come from the lab script.
Errors and limits
Known cases test only what happened before; new attacker timings may still be missed.
Exercises
- EX088 — Lab outcome. Which account was detected and which missed, and why? Rubric: 3 = s2047 detected (6 min); s3310 missed (149 min outside 60); 2 = partial; 1 = wrong; 0 = none.
- EX089 — Two fixes. What two changes were made? Rubric: 3 = 6-hour window; stand-alone keyword-deletion rule; 2 = one; 1 = vague; 0 = none.
- EX090 — Stand-alone logic. Why can a keyword-deletion rule alert on its own? Rubric: 3 = such rules are rare and suspicious regardless of sign-in; 2 = partial; 1 = no reason; 0 = none.
Synthesis
Testing against known cases exposes timing assumptions; fix them and record the limits that remain.
Continue studying — chapter 30 of 40
← Chapter 29 (draft) · Course chapter index · Chapter 31 (draft) →