Chapter 28 — Security operations and blue team: implementation I
Lesson map
Lesson 28 published. Map, example, EX082.
flowchart TD
N0["Learning outcomes"]
N1["28.1 From design to build"]
N2["28.2 Environments"]
N3["28.3 Configuration as code"]
N4["28.4 Parser updates"]
N5["28.5 Change management"]
N6["28.6 Evidence"]
N7["28.7 Pairing"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX082"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["28.1 From design to build"]
N2["28.2 Environments"]
N3["28.3 Configuration as code"]
N4["28.4 Parser updates"]
N5["28.5 Change management"]
N6["28.6 Evidence"]
N7["28.7 Pairing"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX082"]
[Learning outcomes]
|
v
[28.1 From design to build]
|
v
[28.2 Environments]
|
v
[28.3 Configuration as code]
|
v
[28.4 Parser updates]
|
v
[28.5 Change management]
|
v
[28.6 Evidence]
|
v
[28.7 Pairing]
|
v
[Worked example] --> [Exercise EX082]
Learning outcomes
You will (1) implement a detection and its pipeline changes; (2) use staged environments; (3) manage configuration as code; (4) record implementation evidence.
28.1 From design to build
Implementation turns R-07 and the playbook into working parts: load-balancer log forwarding, parser updates, the rule, the alert routing and the automation.
R-07 is the requirement identifier for possible-takeover detection, and keeping that label through implementation lets each component point back to why it exists. Building the parts in dependency order matters: the rule cannot be tested until the parser produces the fields it needs, and routing cannot be checked until the rule creates alerts. Amara draws up a short build plan with that order.
28.2 Environments
Changes go first to a test SIEM with replayed synthetic data, then to production in monitor-only mode, then to alerting. Each stage has entry and exit criteria.
Entry criteria say what must be true before a change enters a stage, such as passing unit tests. Exit criteria say what must be shown before it leaves, such as a week in monitor-only mode with false alerts below the limit. Skipping stages saves days but risks discovering problems in production, where they affect analysts and customers.
28.3 Configuration as code
Parser configurations, rules and routing are stored in version control and deployed by a pipeline, not edited by hand in consoles. This makes changes reviewable and reversible.
Hand edits in a console leave no clear record of who changed what and why, and they are easy to make by mistake. A pipeline applies the same reviewed configuration every time. If a change causes problems, reverting the commit and redeploying restores the previous state in minutes, with a full record of what happened.
28.4 Parser updates
The load-balancer logs use a new format. Amara writes a parser, tests it on 1,000 sample lines, and checks that every field maps to the event model.
A thousand lines include normal requests, failures, unusual characters in usernames and a few malformed lines. Amara counts how many lines parse fully, how many partly and how many fail, and checks a sample by hand against the raw text. Fields such as client address must map to the event model's source field, or the rule will group the wrong values.
28.5 Change management
Production changes follow Lindqvist's change process: a ticket, a review, a window and a rollback plan.
The ticket describes the change and its purpose; review checks the risk; the window chooses a time when problems affect the fewest people; rollback says exactly how to undo it. Detection changes are sometimes treated as low-risk, but a parser change can break many rules at once, so they follow the same process as other production changes. Amara's first ticket is reviewed by Paolo and the SIEM administrator.
28.6 Evidence
Implementation records include commits, test results, change tickets and screenshots of monitor-only results. Auditors and future engineers use them.
Months later, someone may ask when the rule went live and whether it was tested. Commits answer what changed; test results answer whether it worked; tickets answer who approved it; monitor-only results answer how it behaved on real data before alerting. Keeping these together saves time in audits and during incident reviews. Amara links all four from the requirement's row in the traceability table, so the trail starts from R-07.
28.7 Pairing
Amara pairs with an analyst during routing configuration so alerts land where analysts expect them.
Engineers think in terms of rules and queues; analysts think in terms of shifts and workloads. The analyst points out that alerts routed to a general queue would be missed at night, when only the on-call queue is watched. Pairing catches that kind of mismatch before deployment, and gives the analyst a sense of ownership of the new alerts. They finish with a test alert that the analyst opens on the on-call screen.
Rollback rehearsal
Before the production change, Amara rehearses rollback in the test SIEM: she deploys the new parser, then restores the previous version from version control and checks that parsing returns to the old behaviour. A rollback never practised is a hope, not a plan. The rehearsal takes ten minutes and is recorded in the change ticket.
Worked example
During the test stage, 12 of 1,000 sample lines fail to parse: they contain an extra field added by a balancer update. Amara extends the parser, adds the 12 lines as test cases and re-runs: 1,000 parse cleanly. Only then does she request the production change.
Synthetic case. Lines and counts are invented.
Errors and limits
Test data may not include every production variant; monitor parse errors after release.
Exercises
- EX082 — Stages. Describe the three implementation stages. Rubric: 3 = test SIEM with replay, production monitor-only, alerting; 2 = two; 1 = one; 0 = none.
- EX083 — Parse failures. What caused the 12 parse failures, and how were they handled? Rubric: 3 = extra field from balancer update; parser extended, lines added as tests; 2 = partial; 1 = ignored; 0 = none.
- EX084 — Configuration as code. Why deploy configuration from version control? Rubric: 3 = reviewable, reversible, consistent; 2 = partial; 1 = style; 0 = none.
Synthesis
Implement through staged environments, version-controlled configuration and recorded evidence.
Continue studying — chapter 28 of 50
← Chapter 27 (draft) · Course chapter index · Chapter 29 (draft) →