Chapter 19 — Security operations and blue team: monitoring
Lesson map
Lesson 19 published. Map, example, EX055.
flowchart TD
N0["Learning outcomes"]
N1["19.1 Continuous monitoring"]
N2["19.2 Frequencies"]
N3["19.3 Capacity"]
N4["19.4 Source health"]
N5["19.5 Coverage"]
N6["19.6 Client reporting"]
N7["19.7 Reviewing noise"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX055"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["19.1 Continuous monitoring"]
N2["19.2 Frequencies"]
N3["19.3 Capacity"]
N4["19.4 Source health"]
N5["19.5 Coverage"]
N6["19.6 Client reporting"]
N7["19.7 Reviewing noise"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX055"]
[Learning outcomes]
|
v
[19.1 Continuous monitoring]
|
v
[19.2 Frequencies]
|
v
[19.3 Capacity]
|
v
[19.4 Source health]
|
v
[19.5 Coverage]
|
v
[19.6 Client reporting]
|
v
[19.7 Reviewing noise]
|
v
[Worked example] --> [Exercise EX055]
Learning outcomes
You will (1) run continuous monitoring across many clients; (2) balance alert volume with analyst capacity; (3) monitor source health and coverage; (4) report monitoring status to clients.
19.1 Continuous monitoring
NIST SP 800-137 describes continuous monitoring as an ongoing programme tied to risk, not a one-off check. Copperleaf monitors client systems 24 hours a day with defined frequencies for different checks.
Tied to risk means the checks follow the scenarios rather than a fixed list of everything a tool can collect. For BrightSmile, ransomware via remote access drives minute-by-minute rules on logons, accounts and renames, while slower changes such as patch levels are checked daily. The programme is reviewed when the scenarios or the client's systems change.
19.2 Frequencies
Rules on logons and file activity run every minute; daily checks cover patch levels and account lists; monthly reports cover inventory and recommendations.
The frequency of each check follows how fast the harm it watches for can happen. On the incident night, three minutes separated the reception logon from the creation of support2, so logon and account rules must run every minute. An overdue patch or a forgotten account stays the same for days, so a daily list catches it without spending compute every minute. Each check's frequency is written on its rule page, so a reviewer can question it against the scenario it serves.
19.3 Capacity
Two analysts cover nights for forty clients. Rules that page at night must be precise; lower-value alerts wait for the morning queue.
With two people watching forty clients, one high-severity incident such as BrightSmile's can take one analyst for an hour or more, leaving the other alone with everything else. A night page that turns out to be noise therefore costs real attention. Only rules with a proven record, such as the sequence rule, are allowed to page at night.
19.4 Source health
Heartbeat checks show agents that stopped reporting. A missing heartbeat on a server is treated as an alert in its own right.
A server that falls silent may have rebooted, lost its network or had its agent stopped by an attacker, and the SIEM cannot tell those apart from the silence alone. Treating the missing heartbeat as an alert forces someone to find out which it was. The heartbeat lab in chapter 20 shows the states Copperleaf uses: OK, stale and missing.
19.5 Coverage
For each client, the share of priority scenarios with tested detections is tracked. BrightSmile's rises from 40% to 80% after the incident work.
BrightSmile has five priority scenarios, so 40% means two had tested detections and 80% means four. The fifth, the scenario without a tested detection, stays visible as a gap in the monthly report. Mei records which scenarios are covered rather than only the percentage, because the same number could hide very different gaps at another client. Coverage counts only detections that were tested on replayed or lab data, so a rule that exists but has never fired in a test does not raise the figure.
19.6 Client reporting
Monthly reports show alerts handled, incidents, open recommendations and coverage in plain language.
For the office manager at BrightSmile, plain language means a sentence such as "none of this month's alerts needed your action" rather than rule names and event IDs. Open recommendations appear with their age, so she can see that the offline backup copy, for example, is still waiting for a decision. Coverage is shown as the number of priority scenarios with a tested detection, which the office manager can follow without knowing how the rules work. Incidents are described in a few sentences each, with what Copperleaf did and what the client was asked to do.
19.7 Reviewing noise
Each week the team reviews the noisiest rules and tunes or retires them.
Noise is measured by how many alerts a rule produced and how many led to action. The worked example shows the method on the antivirus rule, where nine of ten alerts were installers. Tuning changes the condition so the real cases still fire; retiring is for rules that never produce anything worth an analyst's time. The review also records the change and the date, so if a tuned rule later misses something, the team can trace the miss to that decision.
Handover between shifts
Night analysts hand over open alerts, source-health problems and pending client calls to the morning shift with a short written note and a five-minute conversation. Items without an owner after handover are assigned before the night shift leaves.
Seasonal changes
Clinics close over holidays, so normal activity drops. Baselines and business hours are adjusted when clients notify closures.
Worked example
In the weekly noise review, the "AV disabled" rule produced 10 alerts across clients, nine of them software installers briefly pausing antivirus. Mei proposes alerting only if antivirus stays disabled for more than ten minutes; Raj approves after checking the one real case lasted hours.
Synthetic case. Figures are invented.
Errors and limits
Monitoring frequency costs compute and attention; set it by risk.
Exercises
- EX055 — Frequencies. Give the monitoring frequency for rules, daily checks and reports. Rubric: 3 = minute rules; daily patches/accounts; monthly inventory/recommendations; 2 = two; 1 = one; 0 = none.
- EX056 — Missing heartbeat. Why treat a missing heartbeat as an alert? Rubric: 3 = no monitoring of that host; could be failure or tampering; 2 = partial; 1 = ignore; 0 = none.
- EX057 — AV rule change. How was the AV-disabled rule tuned? Rubric: 3 = alert only if disabled > 10 min; real case lasted hours; 2 = partial; 1 = deleted; 0 = none.
Synthesis
Continuous monitoring balances frequency, capacity and source health, and reports clearly to clients.
Continue studying — chapter 19 of 30
← Chapter 18 (draft) · Course chapter index · Chapter 20 (draft) →