OpenCyber Academy
Rapid course · Lesson 14 · English
Course code: OCA-015Lesson 14: OCA-015-A014

Eleven alerts are not eleven attacks

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 14. Map, example, EX060.

flowchart TD
  A["8 abroad"] --> S["11 alerts"]
  B["2 MFA changes"] --> S
  C["1 forwarding rule"] --> S
  S --> Z["0 attacks"]
  Z --> Q{"Does 11 become a rate?"}
  Q -->|No| K["Keep the parts and the zero"]

The month states a count and a zero. It does not state a rate.

Logical drawing

8 + 2 + 1 = 11
6 + 2 = 8
attacks = 0
8 abroad = 6 Spain holiday + 2 roaming
rate such as 0/11: not printed

Learning outcomes

After this chapter, you should be able to:

  1. Add the three alert parts.
  2. Split the eight abroad alerts.
  3. Refuse a rate, including a borrowed 99 percent.

14.1 The parts add up. They do not become a percentage.

The first month produced 11 alerts and 0 attacks. The 11 splits into 8 sign-ins abroad, 2 MFA changes, and 1 forwarding rule. 8 + 2 + 1 is 11. The 8 splits again: 6 were a colleague on holiday in Spain, and 2 were a partner's phone on a roaming network. 6 + 2 is 8. The 2 MFA alerts were new phones. The forwarding alert was a staff member sending firm email to a personal account, against policy. That finding is a policy issue. It is not an attack, and it may have included client data.

Chapter 4 already says 8 of the 11 alerts were sign-ins abroad. This chapter keeps that statement and does not replace it with a fraction such as 0/11 or 8/11. Those fractions are not on the first-month page. Zero attacks is a count of attacks, not a printed success rate.

Alahmadi, Axon, and Martinovic (2022) discuss analyst experience of very high false-positive shares in security operations centres. Their "99%" is that study's figure. It is not Marlow & Finch's rate, and it must not be written onto these 11 alerts. Axelsson (2000) is the base-rate point already used: rare events make most alerts innocent, and the aim is quick checks, not zero alerts. Sommer and Paxson (2010) argue that a closed-world lab result does not transfer to operational detection. These 11 alerts are not a training set. This book fits no classifier to them, and BrightPath publishes no Sigma rule. The built-in alerts stay the provider's alerts.

Lesson table; scroll horizontally to read all columns →

Part Count What it was
Abroad sign-ins 8 6 holiday, 2 roaming
MFA changes 2 New phones
Forwarding 1 Personal mailbox, against policy
Attacks 0 None of the 11

14.2 Worked example

Lesson table; scroll horizontally to read all columns →

Offered summary Arithmetic Decision
"11 alerts, 0 attacks." The month's two headlines Keep
"6 and 2 make the 8." The abroad split Keep
"0 of 11 is the firm's false-positive rate." No rate is printed Refuse
"99% applies to this month." Another study's figure Refuse

Synthetic case. The 11 alerts are invented, and the zero is a count.

Errors and limits

Do not import another paper's percentage as this firm's result.

Apply

  • EX060 — Partition. How do 8, 2, and 1 relate to 11, how do 6 and 2 relate to 8, and how many of the 11 were attacks?
    Show marking criteria
    Rubric: 3 = 8 plus 2 plus 1 is 11; 6 plus 2 is the 8 abroad alerts; attacks were 0; do not print a rate or borrow 99 percent;; 2 = the parts add and the attack count stays 0;; 1 = one part;; 0 = none.
Worked answers

EX060. 8 plus 2 plus 1 is 11. 6 plus 2 is the 8 abroad alerts. None of the 11 were attacks. Do not print a rate or borrow another study's 99 percent.