Chapter 4 — Keeping watch and getting better | Rapid course Security operations and blue team

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 04 published. Map, example, EX039.

flowchart TD
  N0["Learning outcomes"]
  N1["4.1 Ongoing alerts"]
  N2["4.2 Measuring usefulness"]
  N3["4.3 Reducing noise"]
  N4["4.4 Lessons review"]
  N5["4.5 Continuous monitoring"]
  N6["4.6 Next steps"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX039"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["4.1 Ongoing alerts"]
  N2["4.2 Measuring usefulness"]
  N3["4.3 Reducing noise"]
  N4["4.4 Lessons review"]
  N5["4.5 Continuous monitoring"]
  N6["4.6 Next steps"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX039"]

[Learning outcomes]
    |
    v
[4.1 Ongoing alerts]
    |
    v
[4.2 Measuring usefulness]
    |
    v
[4.3 Reducing noise]
    |
    v
[4.4 Lessons review]
    |
    v
[4.5 Continuous monitoring]
    |
    v
[4.6 Next steps]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX039]

Learning outcomes

You will (1) set up simple ongoing monitoring with a provider; (2) measure whether alerts are useful; (3) run a short lessons review; (4) plan the next month and beyond.

4.1 Ongoing alerts

BrightPath turns on alerts in the email service for the four signals from chapter 1 and sends Leo a weekly summary. Shared detection ideas such as SigmaHQ rules inspire the alert choices, though BrightPath uses the email service's built-in alerts.

The alert for new forwarding rules matters most, because it would have caught Elaine's 23:51 rule on Friday night, long before the client's Monday phone call. Built-in alerts suit a firm with no security staff: BrightPath does not have to write or maintain its own rules, only check what the email service sends.

4.2 Measuring usefulness

In the first month, there are 11 alerts: 8 for sign-ins abroad (6 were Leo's colleague on holiday in Spain, 2 a partner's phone using a roaming network), 2 MFA changes (new phones), 1 forwarding rule (a staff member forwarding to her own personal email, against policy). None were attacks, but the forwarding rule found a policy issue.

Every alert was checked against a simple question, "is there an innocent explanation we can confirm?", and the answer took minutes: a holiday in Spain, a roaming phone, two new phones. The forwarding rule shows the value of the signal even without an attack: a staff member was sending firm email, possibly including client data, to a personal account.

4.3 Reducing noise

Staff tell Leo before travelling; BrightPath adds the trip to a list so alerts are checked quickly. Axelsson (2000) explains why rare attacks make most alerts false; the aim is quick checks, not zero alerts.

In the first month, 8 of the 11 alerts were sign-ins abroad and all had innocent explanations. With the travel list, a sign-in from Spain during a colleague's known holiday can be checked in a minute. The forwarding-rule and MFA alerts stay unchanged, because they are rare and each one needs a look.

4.4 Lessons review

A 30-minute meeting asks what happened, what worked and what to change. Actions: MFA for all staff (done), phone-check rule for bank detail changes (done), staff phishing training (booked), and a twice-yearly check of who has admin rights.

Each action ties to a gap the incident exposed. MFA for all staff, because Elaine's account had none. A phone-check rule, because a client paid £4,800 on the strength of an email alone. Phishing training, because Elaine entered her password on a fake page. The admin-rights check, because a mailbox with admin rights could have done far more harm.

4.5 Continuous monitoring

NIST SP 800-137 (Dempsey et al., 2011) describes monitoring as continuous, feeding decisions over time. For Leo this means the weekly summary, the monthly partner review and yearly checks of the plan.

Each of the three timescales has a job. The weekly summary shows Leo the alerts and how each was explained. The monthly review lets the partners see trends, such as the report button being used more. The yearly check asks whether the four signals and the call list still fit the firm, for example after new staff or a change of IT provider.

4.6 Next steps

Leo plans to learn more about incident response and resilience.

Each topic builds on this month's work. Incident response would turn BrightPath's mailbox playbook into a short set of playbooks for other events, such as a lost laptop or a ransomware note. Resilience starts from the firm's backups, the one part of the plan that existed before Elaine's incident, and asks whether the accounts software and client files could be restored quickly after an attack. Leo would test that by restoring one client folder.

Worked example

Two months later, a staff member reports a fake "invoice overdue" email. BrightPath confirms no one entered a password; the sender is blocked. The report button is used three more times that month. Leo counts this as success: people are watching too.

Synthetic case. Alerts and reports are invented.

Errors and limits

A small firm relies heavily on its provider; check the provider's work through the weekly summaries.

Exercises

EX039 — Alert count. How many alerts in the first month and how many attacks?
Show marking criteria
Rubric: 3 = 11; none; 2 = one; 1 = vague; 0 = none.
EX040 — Sign-in alerts. Explain the 8 sign-in alerts.
Show marking criteria
Rubric: 3 = 6 holiday in Spain; 2 roaming phone; 2 = partial; 1 = vague; 0 = none.
EX041 — Policy finding. What did the forwarding alert find?
Show marking criteria
Rubric: 3 = staff forwarding to personal email; policy issue; 2 = partial; 1 = vague; 0 = none.
EX042 — Travel list. How was travel noise reduced?
Show marking criteria
Rubric: 3 = staff tell Leo; trip list for BrightPath; 2 = partial; 1 = vague; 0 = none.
EX043 — Realistic aim. What is the aim with alerts?
Show marking criteria
Rubric: 3 = quick checks, not zero alerts; 2 = partial; 1 = vague; 0 = none.
EX044 — Lessons actions. List three actions from the lessons review.
Show marking criteria
Rubric: 3 = MFA all; phone-check bank changes; training; admin check (three); 2 = two; 1 = one; 0 = none.
EX045 — Review length. How long was the lessons meeting?
Show marking criteria
Rubric: 3 = 30 minutes; 2 = one; 1 = vague; 0 = none.
EX046 — Continuous meaning. What does continuous monitoring mean for Leo?
Show marking criteria
Rubric: 3 = weekly summary, monthly review, yearly plan check; 2 = partial; 1 = vague; 0 = none.
EX047 — Later report. What happened with the later phishing report?
Show marking criteria
Rubric: 3 = no password entered; sender blocked; 2 = partial; 1 = vague; 0 = none.
EX048 — Success measure. Why did Leo count reports as success?
Show marking criteria
Rubric: 3 = people watching too; 2 = partial; 1 = vague; 0 = none.
EX049 — Provider check. How does Leo check the provider?
Show marking criteria
Rubric: 3 = weekly summaries; 2 = partial; 1 = vague; 0 = none.
EX050 — Admin rights. How often are admin rights checked?
Show marking criteria
Rubric: 3 = twice a year; 2 = one; 1 = vague; 0 = none.

Synthesis

Keep watching a few signals, measure them, learn from each event and keep people involved.

Continue studying — chapter 04 of 04

← Chapter 03 (draft) · Course chapter index · Worked results ↓ · Next course via chapter index →