OpenCyber Academy
Master's · Chapter 2 · English
Course code: OCA-172Chapter 02: OCA-172-C002

Chapter 2 — Network and communications security: advanced seminar

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 02 published. Map, example, EX008.

flowchart TD
  N0["Learning outcomes"]
  N1["2.1 How detection works"]
  N2["Lab — The base-rate problem"]
  N3["2.2 Reading the base rates"]
  N4["2.3 Triage in practice"]
  N5["Flow records as a second view"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX008"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["2.1 How detection works"]
  N2["Lab — The base-rate problem"]
  N3["2.2 Reading the base rates"]
  N4["2.3 Triage in practice"]
  N5["Flow records as a second view"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX008"]

[Learning outcomes]
    |
    v
[2.1 How detection works]
    |
    v
[Lab — The base-rate problem]
    |
    v
[2.2 Reading the base rates]
    |
    v
[2.3 Triage in practice]
    |
    v
[Flow records as a second view]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX008]

Learning outcomes

You will (1) explain how network intrusion detection works; (2) analyse the base-rate problem; (3) discuss alert triage practice.

2.1 How detection works

Network intrusion detection systems inspect traffic or flow records and raise alerts when patterns match signatures or deviate from expected behaviour. NIST SP 800-94 distinguishes signature-based, anomaly-based and stateful protocol analysis methods. Signatures are precise for known patterns but miss novel ones; anomaly methods can find new patterns but produce more false alarms. Paxson's Bro, now Zeek, emphasised policy scripts that describe what the operator cares about, a philosophy that fits Amara's tuning focus.

For Amara, the practical consequence is that each rule's false alarms on Corvane's traffic must be measured rather than assumed from its description.

Understanding this explains why tuning matters.

Lab — The base-rate problem

"""Precision of an alert rule at different base rates (Axelsson's point), synthetic numbers (stdlib only)."""
tpr, fpr, events = 0.95, 0.01, 1_000_000
for prev in (0.1, 0.01, 0.001, 0.0001):
    tp = tpr * prev * events; fp = fpr * (1 - prev) * events
    print(f"base rate {prev:<7} true alerts {tp:9.0f} false alerts {fp:8.0f} precision {tp/(tp+fp):.3f}")

Run with python3 lab_baserate.py. Verified output:

base rate 0.1     true alerts     95000 false alerts     9000 precision 0.913
base rate 0.01    true alerts      9500 false alerts     9900 precision 0.490
base rate 0.001   true alerts       950 false alerts     9990 precision 0.087
base rate 0.0001  true alerts        95 false alerts     9999 precision 0.009

2.2 Reading the base rates

With a rule that detects 95 percent of malicious events and falsely alerts on only 1 percent of benign events, precision is 0.913 when one event in ten is malicious, but only 0.009 when one in ten thousand is. Axelsson (2000) made exactly this point: in real networks malicious events are rare, so even a good detector produces mostly false alarms. Tuning that lowers the false positive rate on benign traffic therefore matters more than small gains in detection rate.

At one malicious event in ten thousand, the rule raises roughly a hundred false alerts for every true one.

2.3 Triage in practice

In seminar, Amara compares triage practices from published guidance and partner interviews. Effective teams group related alerts, enrich them with asset and owner information, and document why alerts are closed. Weak practice closes alerts without notes, which loses the information needed to tune rules. She notes that closure notes are also her best source of labels, linking triage quality to research quality.

Triage means deciding quickly which alerts need investigation. Analysts check the rule, the hosts involved and recent context, then close, escalate or investigate. Time pressure makes bulk closing tempting, which is the behaviour Amara's study aims to reduce.

Flow records as a second view

Signatures inspect packet content, but much of Corvane's traffic is encrypted with TLS, which hides payloads from the sensors. Flow records, standardised as IPFIX in RFC 7011, summarise each connection by addresses, ports, timing and byte counts without content. Amara notes that flow data remain useful when payloads are encrypted, for example to see that a backup server transfers large volumes at night, and that this context helps judge alerts. It also protects privacy better than full packet capture, since it records who talked to whom rather than what was said.

Worked example

Amara presents the base-rate table and shows that Corvane's estimated prevalence puts them close to the 0.001 row, explaining why analysts see mostly false alerts.

Synthetic case. Rates are synthetic.

Errors and limits

Quoting a detector's accuracy without the base rate misleads decision makers.

Exercises

Synthesis

Low base rates make false positive control the central problem of alert quality.

Continue studying — chapter 02 of 08

← Chapter 01 (draft) · Course chapter index · Chapter 03 (draft) →

Worked answers

EX008. Signatures precisely match known patterns but miss new ones. Anomaly methods can catch new behaviour but raise more false alarms.

EX009. 0.490 at 0.01 and 0.009 at 0.0001.

EX010. With rare malicious events, false positives dominate, so reducing them improves precision far more.

EX011. They record why alerts were benign or real, providing labels and tuning evidence.

EX012. It shows which system and owner are involved, so analysts judge importance quickly.

EX013. Policy scripts expressing what the operator cares about.

EX014. At that base rate, roughly nine in ten alerts are false.