Chapter 12 — Detection engineering basics
Lesson map
Lesson 12 published. Map, example, EX045.
flowchart TD
N0["Learning outcomes"]
N1["12.1 The cycle"]
N2["12.2 Choosing a behaviour"]
N3["12.3 Structure"]
N4["12.4 Testing"]
N5["12.5 Documentation"]
N6["12.6 Review"]
N7["Rule naming"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX045"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["12.1 The cycle"]
N2["12.2 Choosing a behaviour"]
N3["12.3 Structure"]
N4["12.4 Testing"]
N5["12.5 Documentation"]
N6["12.6 Review"]
N7["Rule naming"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX045"]
[Learning outcomes]
|
v
[12.1 The cycle]
|
v
[12.2 Choosing a behaviour]
|
v
[12.3 Structure]
|
v
[12.4 Testing]
|
v
[12.5 Documentation]
|
v
[12.6 Review]
|
v
[Rule naming]
|
v
[Worked example] --> [Exercise EX045]
Learning outcomes
You will (1) explain the detection engineering cycle; (2) write a detection idea in plain language and Sigma-style structure; (3) test rules with synthetic events; (4) document rules for analysts.
12.1 The cycle
Detection engineering turns knowledge about threats into rules: pick a behaviour, find the data that shows it, write the rule, test it, deploy it, then measure and tune. It is a loop, not a one-off task. For Carlos the cycle starts from INC-0417, a behaviour the SOC has now seen. The loop matters because the first version of a rule is rarely right; the live month later shows one benign and one real firing, and measuring those is what tells him whether to tune.
12.2 Choosing a behaviour
After INC-0417, Aisha asks Carlos to write a rule for "successful logon from a source that had many failures shortly before". The behaviour matches ATT&CK's brute-force technique and is visible in portal logs. Choosing a behaviour seen in a real case gives the rule a known test: it must fire on the Monday log. Portal logs already hold the needed fields, source, user, result and time, so no new data collection is required before writing the rule. The rule's documentation page links back to INC-0417 so reviewers can see the original pattern.
12.3 Structure
Sigma rules have a title, description, log source, detection condition, false-positive notes and level. Writing the idea in this structure, even for a SIEM that uses its own language, forces clarity: Carlos writes log source "remote-access portal authentication", condition "success where same source had 20 or more failures in previous 15 minutes", level high. The 15-minute window is wider than the original failure rule's ten minutes, so a slower guesser is still linked to the success. The structure also records expected false positives before the rule goes live.
12.4 Testing
Carlos replays the synthetic Monday log in a test index. The rule fires once, at 09:09, for parks01. He also replays a week of normal logs: it does not fire. Testing the rule both ways, against attack and normal logs, matters. Firing on the Monday log shows the rule catches what it was built for, while staying silent on a normal week shows it will not flood the queue. A rule tested only on the attack would give no idea of its false positives. Both replays are saved with the rule as its test record.
12.5 Documentation
Each rule gets a short page: what it detects, why, data needed, expected false positives and first triage steps. The page lets an analyst who has never seen the rule handle its first alert. In the first live month, the forgotten-password case was closed in two minutes because the triage steps said to check whether the user could be reached and confirm. Without documentation, the same alert might take a new analyst much longer to close. The page is linked from the alert itself, so the triage steps are one click away.
12.6 Review
Another analyst reviews logic and documentation before deployment. A second reader catches mistakes such as a window set to 15 seconds instead of minutes, or a condition that never matches. The review also checks the documentation is clear to someone who did not write the rule. Carlos's rule passed this review before going live on the council's SIEM. The reviewer's name and the review date are both recorded on the rule's page. Changes after review need the same check again before deployment.
Rule naming
Rules use clear names that state the behaviour so analysts recognise them in a busy queue.
Worked example
In the first month live, the rule fires twice: once for a staff member who forgot a new password 25 times from home and then got it right (benign, closed in two minutes thanks to the triage steps) and once for a real guessing attempt against a councillor's account, stopped because MFA had since been enabled.
Synthetic case. Rules and events are synthetic.
Errors and limits
A rule tested on one replay may still miss variations, such as guessing spread across many sources.
Exercises
- EX045 — Cycle steps. List the steps of the detection cycle. Rubric: 3 = behaviour, data, write, test, deploy, measure/tune; 2 = four; 1 = two; 0 = none.
- EX046 — Sigma fields. Name four fields of a Sigma rule. Rubric: 3 = title, description, log source, detection, false positives, level (four); 2 = two; 1 = one; 0 = none.
- EX047 — Test results. What did testing show? Rubric: 3 = fired once at 09:09; not on normal week; 2 = partial; 1 = vague; 0 = none.
- EX048 — Live firings. Describe the two live firings. Rubric: 3 = forgotten password benign; real guessing on councillor stopped by MFA; 2 = partial; 1 = one; 0 = none.
Synthesis
Detection engineering is a tested, documented, reviewed loop.
Continue studying — chapter 12 of 18
← Chapter 11 (draft) · Course chapter index · Chapter 13 (draft) →