Chapter 5 — Triage and the alert pipeline
Lesson map
Lesson 05 published. Map, example, EX021.
flowchart TD
N0["Learning outcomes"]
N1["5.1 The pipeline"]
N2["5.2 Priorities"]
N3["5.3 Service levels"]
N4["5.4 Triage notes"]
N5["5.5 Unowned queues"]
N6["5.6 New hire"]
N7["Closing well"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX021"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["5.1 The pipeline"]
N2["5.2 Priorities"]
N3["5.3 Service levels"]
N4["5.4 Triage notes"]
N5["5.5 Unowned queues"]
N6["5.6 New hire"]
N7["Closing well"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX021"]
[Learning outcomes]
|
v
[5.1 The pipeline]
|
v
[5.2 Priorities]
|
v
[5.3 Service levels]
|
v
[5.4 Triage notes]
|
v
[5.5 Unowned queues]
|
v
[5.6 New hire]
|
v
[Closing well]
|
v
[Worked example] --> [Exercise EX021]
Learning outcomes
You will (1) design an alert pipeline from SIEM to decision; (2) set triage priorities using asset value; (3) define service levels; (4) avoid alert queues that nobody owns.
5.1 The pipeline
Alerts flow from the SIEM to the provider's queue at night and to Fenwick's analyst by day. Each alert is acknowledged, triaged, then closed, escalated or turned into an incident. Every stage has an owner; March showed what happens when nobody owns the queue. Splitting the queue by time of day gives Fenwick cover around the clock without a night shift of its own. The handover at 08:00 keeps the two halves connected, so an alert raised at night is not lost when the provider's shift ends. Tom checks each morning that every night alert has a note and an outcome.
5.2 Priorities
Priority combines rule severity with asset value from the asset list built in chapter 3. An admin change on a domain controller is critical; a macro document opened on a reception PC is low. Priority tells the provider and Tom which alerts need attention first. Using asset value means the same rule can produce different priorities: an admin change on a domain controller could give an attacker control of every account, while a macro document on a reception PC affects one machine. Priya reviews priorities each month with Tom and the provider, and adjusts them when the asset list changes.
5.3 Service levels
Critical: acknowledge in 15 minutes and call Priya. High: 30 minutes. Medium: same working day. Low: weekly review. These come from the board's first-year goal. Each level has a target that can be measured and reported. Critical alerts get the shortest target because March showed how quickly damage can be done: fifteen minutes passed between the admin change and its removal. The provider and Tom record acknowledgement times, and Priya reports them monthly to the risk committee. Missed targets are discussed at the monthly review with the provider, with the reason and any fix recorded.
5.4 Triage notes
Every decision has a note: what was seen, checked, decided and why. The provider's notes must be readable by Fenwick's analyst the next morning. Good notes let a second analyst understand a decision without repeating the work, and show later whether a closed alert was handled correctly. Tom reviews the provider's night notes each morning and asks about any that are unclear. He keeps a short guide with examples of good notes, which the provider uses to train its analysts on Fenwick's service. Notes are also used in the monthly review, to check precision and to find rules that need tuning.
5.5 Unowned queues
Priya removes the old shared mailbox; alerts now go only to queues with named owners and service levels. In March, the alert went to a shared mailbox that nobody checked at night, which is why it sat unseen for nine hours. Removing the mailbox means every alert has a named owner who is accountable for acknowledging it within its service level. Priya checks that no rule still sends to the old address. Any new queue must have an owner and a service level before rules can send to it.
5.6 New hire
Fenwick hires Tom Reyes as its first analyst. He owns daytime triage and reviews night notes each morning. Hiring an analyst gives Fenwick in-house knowledge that the provider cannot supply, such as which finance staff use macros or when IT runs maintenance. Tom's morning review keeps the night and day halves of the queue connected. He also maintains rule documentation and runbooks. Priya remains SOC owner and decides on escalation and business matters, while Tom handles daily operations. Their roles are written into the charter, so the provider knows to call Priya for decisions and Tom for questions about a night note.
Closing well
A closed alert states the evidence for closure, such as a change ticket number.
Weekend cover
At weekends the provider covers all hours, calling on-call IT staff.
Worked example
In week three, the provider calls Priya at 23:10 about a critical alert: a new-country VPN login for a sales manager. Priya confirms the manager is at a trade fair abroad, as shown in the travel calendar the provider did not have. The alert is closed, and Priya arranges for the provider to receive the travel calendar.
Synthetic case. Alerts and people are invented.
Errors and limits
Service levels can push analysts to close quickly; review closure quality as well as speed.
Exercises
- EX021 — Pipeline stages. Name the stages of the alert pipeline. Rubric: 3 = acknowledge, triage, close/escalate/incident; 2 = partial; 1 = vague; 0 = none.
- EX022 — Critical service level. What is the service level for critical alerts? Rubric: 3 = acknowledge 15 min; call Priya; 2 = partial; 1 = vague; 0 = none.
- EX023 — Mailbox decision. Why remove the shared mailbox? Rubric: 3 = nobody owned it; March missed; 2 = partial; 1 = vague; 0 = none.
- EX024 — Trade fair call. How was the week-three alert resolved? Rubric: 3 = travel calendar showed fair; closed; calendar shared with provider; 2 = partial; 1 = vague; 0 = none.
- EX025 — Note purpose. Why must night notes be readable by Tom? Rubric: 3 = continuity across provider and in-house; 2 = partial; 1 = vague; 0 = none.
Synthesis
Every alert needs an owner, a priority and a service level.
Continue studying — chapter 05 of 12
← Chapter 04 (draft) · Course chapter index · Chapter 06 (draft) →