OpenCyber AcademyLearning outcomes
You will (1) turn a practical SOC problem into a researchable question; (2) run a structured literature search; (3) appraise sources by type and strength; (4) keep a reproducible search log.
1.1 From problem to question
Tariq Haddad, a master's student at the fictional Ashworth Institute of Technology, works part-time in the corporate SOC of the fictional Pelham Water Utility. Analysts complain of "too many alerts". That is a problem, not a question. With supervisor Dr. Ingrid Solberg, Tariq narrows it: "Does adding reviewed allow-lists to the ten noisiest detection rules reduce weekly alert volume without suppressing true attacks, compared with untuned rules over the same weeks?"
The second half of the question, "without suppressing true attacks", is what makes it a security question rather than a workload one.
1.2 Parts of a good question
The question names an intervention (allow-lists), a population (ten rules, one SOC), a comparison (untuned rules), outcomes (volume and suppressed attacks) and a time frame. Each part can be measured.
Each part also limits the claim. "Ten rules, one SOC" means the result describes Pelham's noisiest rules, not detection tuning in general. "Untuned rules over the same weeks" means anything that affected all rules, such as a holiday, can be subtracted. Naming two outcomes means a fall in volume cannot be reported without the suppressed-attack count beside it.
1.3 Searching the literature
Tariq searches digital libraries (ACM, IEEE, USENIX) and grey literature (NIST, MITRE) with recorded terms: "alert fatigue", "false positive" AND "security operations", "detection tuning", "SOC" AND "qualitative". He records dates, databases and hit counts.
His ACM search on 3 February 2026 returned 214 hits; 31 were screened by abstract and 9 included. The exclusions are logged with reasons: 12 network-IDS algorithm papers without analyst data and 10 duplicates. Grey literature is searched separately so that NIST and MITRE documents are not mixed with empirical studies in the counts.
1.4 Appraising sources
Peer-reviewed empirical studies, such as Alahmadi, Axon and Martinovic (2022) on benign triggers, carry more weight than vendor blogs. Frameworks — NIST SP 800-92 on log management, the SigmaHQ rule format — define practice but are not evidence of effect. Tariq tags each source by type.
The tags shape what each source can support in the dissertation. Alahmadi et al. can be cited for how analysts experience alarms; SP 800-92 can be cited for what good log management looks like; neither can show that allow-lists reduce volume without hiding attacks. That claim needs Tariq's own measurement.
1.5 What the literature says
Analysts report most alarms are benign triggers (Alahmadi et al., 2022); low base rates make false alarms dominate (Axelsson, 2000); mismatched incentives distort SOC metrics (Kokulu et al., 2019). Little published work measures tuning effects with a comparison group.
The last sentence is Tariq's gap. Studies describe why alerts are noisy and why metrics mislead, but few compare tuned rules with untuned ones over the same period. His seminar claim builds on this: volume reduction alone is a misleading success metric, while Alahmadi et al.'s finding that volume drives burnout is the counter-evidence that reduction still has value.
1.6 The search log
The log lets a reader rerun the search and see why sources were included or excluded.
Tariq's entry for the ACM search shows what that requires: the date, the database, the exact string, the year range, hit counts at each stage and the reasons for exclusion. A reader who disagrees with excluding the 12 network-IDS algorithm papers can see the decision and judge it, rather than suspecting that inconvenient papers vanished. Dr. Solberg reviews the log before the literature chapter is drafted.
Worked example
Tariq's search log entry: "2026-02-03, ACM DL, ('alert fatigue' OR 'false positive') AND ('security operations' OR SOC), 2010–2026: 214 hits; 31 screened by abstract; 9 included. Exclusions: 12 network-IDS algorithm papers without analyst data; 10 duplicates."
Synthetic case. Ashworth, Pelham Water Utility and all people are invented.
Errors and limits
Searches miss industry reports behind paywalls; note this as a limitation.
Exercises
- EX001 — Problem versus question. Why is "too many alerts" not yet a research question?
Show marking criteria
Rubric: 3 = no intervention, comparison, measurable outcome or time frame; 2 = partial; 1 = it is; 0 = none. - EX002 — Question parts. Name the five parts of Tariq's question.
Show marking criteria
Rubric: 3 = intervention, population, comparison, outcomes, time; 2 = three; 1 = one; 0 = none. - EX003 — Search record. What does Tariq record for each search?
Show marking criteria
Rubric: 3 = date, database, terms, hits, screening counts; 2 = three; 1 = one; 0 = none. - EX004 — Source weight. Compare the evidential weight of Alahmadi et al. and a vendor blog.
Show marking criteria
Rubric: 3 = peer-reviewed empirical vs unreviewed marketing; 2 = partial; 1 = equal; 0 = none. - EX005 — Frameworks. What role do NIST SP 800-92 and Sigma play in the review?
Show marking criteria
Rubric: 3 = define practice/format, not evidence of effect; 2 = partial; 1 = evidence; 0 = none. - EX006 — Gap. State the gap Tariq's study addresses.
Show marking criteria
Rubric: 3 = few measurements of tuning effects with comparison group; 2 = partial; 1 = vague; 0 = none. - EX007 — Exclusions. Why were network-IDS algorithm papers excluded?
Show marking criteria
Rubric: 3 = no analyst/operational data relevant to the question; 2 = partial; 1 = random; 0 = none.
Synthesis
A researchable question has measurable parts; a logged, appraised literature search shows where it fits.