Chapter 1 — Research question and literature method
Lesson map
Lesson 01 published. Map, example, EX001.
flowchart TD
N0["Learning outcomes"]
N1["1.1 The case"]
N2["1.2 Literature method"]
N3["1.3 Defining the measures"]
N4["1.4 Scope and boundaries"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX001"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["1.1 The case"]
N2["1.2 Literature method"]
N3["1.3 Defining the measures"]
N4["1.4 Scope and boundaries"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX001"]
[Learning outcomes]
|
v
[1.1 The case]
|
v
[1.2 Literature method]
|
v
[1.3 Defining the measures]
|
v
[1.4 Scope and boundaries]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX001]
Learning outcomes
You will (1) frame a master's question about intrusion detection alert quality; (2) search and screen literature systematically; (3) define measures.
1.1 The case
Amara Okoye is a master's student at the fictional Easterly University, supervised by Dr Felix Brandt. Her industry partner is the fictional Corvane Logistics, whose small security team receives over four thousand network intrusion detection alerts a week from sensors at its two data centres. Analysts admit they skim most alerts and close them in bulk. Amara's dissertation asks a focused question: can documented tuning of the alert rules cut alert volume substantially while keeping nearly all detections of the events the team cares about? The question matters because an analyst who ignores most alerts will eventually ignore an important one.
1.2 Literature method
Amara searches two digital libraries and the NIST publications site with fixed search strings, records the number of results, and screens titles and abstracts against written inclusion criteria: empirical or authoritative work on network intrusion detection, alert quality or tuning. She keeps a log of every decision so another student could repeat the search. Core sources emerge quickly: NIST SP 800-94 on intrusion detection and prevention systems, Axelsson (2000) on the base-rate fallacy, Paxson's 1999 description of Bro and Sommer and Paxson (2010) on the limits of machine learning in this domain.
1.3 Defining the measures
Amara chooses three measures. Alert volume is alerts per week, normalised by traffic so that quieter weeks do not look like improvement. Precision is the share of alerts that correspond to events the team would act on. Recall is the share of those actionable events that produce at least one alert. Recall needs a labelled set of known events, which she builds with the team from a week of past incidents and authorised test activity recorded in their change logs, never by generating attacks herself.
1.4 Scope and boundaries
The study covers rule tuning only: suppressing known benign sources, adjusting thresholds and disabling rules for services Corvane does not run. It does not evaluate new detection products or machine learning. All observation uses alert exports and labelled records provided by Corvane under a data agreement; Amara does not run traffic against their sensors.
Amara's scope covers alert rules on Corvane's two sensors over a fixed period, using synthetic replays where real data cannot leave the company. It excludes endpoint tools and the response process after triage. Stating these boundaries prevents the dissertation from overclaiming and tells readers which questions remain open.
Worked example
Amara's question statement, search log and three measure definitions fit on two pages and are approved by Dr Brandt and Corvane's security lead before data collection.
Synthetic case. Amara Okoye, Easterly University, Corvane Logistics and all figures are synthetic.
Errors and limits
Measuring only alert volume rewards switching rules off; recall must be measured alongside it.
Exercises
- EX001 — Why tuning matters. Explain why bulk-closing alerts is a security risk. Rubric: 3 = important alerts ignored among noise;; 2 = risk;; 1 = vague;; 0 = none.
- EX002 — Search log. What should a literature search log record? Rubric: 3 = strings, sources, dates, counts, decisions;; 2 = some;; 1 = vague;; 0 = none.
- EX003 — Normalising volume. Why normalise alert volume by traffic? Rubric: 3 = traffic changes confound;; 2 = fairness;; 1 = vague;; 0 = none.
- EX004 — Precision and recall. Define precision and recall for alert rules. Rubric: 3 = both correct;; 2 = one;; 1 = vague;; 0 = none.
- EX005 — Labelled set source. How is the labelled event set built without attacking anything? Rubric: 3 = past incidents and authorised tests from records;; 2 = records;; 1 = attacks;; 0 = none.
- EX006 — Volume-only danger. What perverse outcome does a volume-only measure reward? Rubric: 3 = switching rules off;; 2 = fewer alerts;; 1 = vague;; 0 = none.
- EX007 — Scope boundary. Name two things outside the study's scope. Rubric: 3 = new products and ML;; 2 = one;; 1 = vague;; 0 = none.
Synthesis
A precise question, a logged literature method and paired volume, precision and recall measures frame the study.