◇ OpenCyber Academy
Engineering · Chapter 42 · English

Chapter 42 — Security operations and blue team: applied research

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 42 — Security operations and blue team: applied research

Lesson map

Lesson 42 published. Map, example, EX124.

flowchart TD
  N0["Learning outcomes"]
  N1["42.1 Research in practice"]
  N2["42.2 The question"]
  N3["42.3 Literature"]
  N4["42.4 Method"]
  N5["42.5 Ethics"]
  N6["42.6 Results"]
  N7["42.7 Decision"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX124"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["42.1 Research in practice"]
  N2["42.2 The question"]
  N3["42.3 Literature"]
  N4["42.4 Method"]
  N5["42.5 Ethics"]
  N6["42.6 Results"]
  N7["42.7 Decision"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX124"]

[Learning outcomes]
    |
    v
[42.1 Research in practice]
    |
    v
[42.2 The question]
    |
    v
[42.3 Literature]
    |
    v
[42.4 Method]
    |
    v
[42.5 Ethics]
    |
    v
[42.6 Results]
    |
    v
[42.7 Decision]
    |
    v
[Worked example] --> [Exercise EX124]

Learning outcomes

You will (1) frame an engineering research question from SOC data; (2) review relevant studies; (3) run a small, ethical study inside the organisation; (4) report results with limits.

42.1 Research in practice

Engineers often need evidence their organisation lacks. Applied research answers a local question with a careful method, borrowing ideas from published studies.

Published studies rarely match a particular SOC's data, tools and customers. Applied research asks a narrow question that matters locally, uses data the organisation already holds and states its limits honestly. Borrowing methods from published work, such as how to label alerts or measure outcomes, gives the local study a stronger foundation. Results are written up so others can check them.

42.2 The question

Amara asks: "Does adding device continuity (same device as usual) to takeover alerts reduce false escalations without missing real takeovers?" It comes from her queue day in chapter 37.

During her queue days, Amara noticed that many takeover alerts turned out to be customers logging in from their usual phone after mistyping passwords. Device continuity, whether the device matches the customer's history, might separate these from real takeovers. Framing the question with both benefits and risks keeps the study balanced. She writes the question down first.

42.3 Literature

Alahmadi et al. (2022) argue alarms should be contextual; Hassan et al. (2019) show context from provenance can rank alerts. Sommer and Paxson (2010) caution that patterns learned in one environment may not transfer.

The literature supports the idea that context helps analysts judge alerts, which justifies testing device continuity. The caution about transfer matters too: results found at Lindqvist may not hold at another company, and patterns may shift over time even at Lindqvist. Amara uses these sources to frame her method and limits.

42.4 Method

Amara labels three months of takeover alerts (82 in total) as real or benign using case outcomes, then checks how device continuity would have changed the decision. She pre-states her measure: escalations avoided and real takeovers that would have been de-prioritised.

Stating the measure before looking at the data prevents choosing whichever result looks best afterwards. Case outcomes give labels grounded in investigation rather than guesses. Asking how device continuity would have changed each decision is a retrospective test, which avoids putting customers at risk while the idea is evaluated.

42.5 Ethics

The study uses existing case records; customer identifiers are removed before analysis, and the privacy officer approves the use.

Even though the records already exist, using them for research is a new purpose that needs approval. Removing identifiers before analysis means Amara does not see which customer each alert concerned. The privacy officer checks that the purpose is legitimate and that results will be reported only in aggregate. The approval and its conditions are filed with the study, and the de-identified data set is deleted once the results have been reviewed.

42.6 Results

Device continuity would have de-prioritised 29 of 70 benign alerts and 1 of 12 real takeovers. The missed case involved a customer whose phone had been stolen.

De-prioritising 29 of 70 benign alerts would save real analyst time. Missing one of twelve real takeovers is a serious cost, because that customer suffered harm. The stolen-phone case shows the weakness: an attacker using the victim's own device looks continuous by design. The result shows why a single signal should not decide alone. Amara reports both counts together so neither the saving nor the miss is hidden.

42.7 Decision

The team adds device continuity as information on the alert, not as an automatic de-prioritiser, and logs analyst decisions for review.

Showing device continuity as information lets analysts use it while still judging each case, so the stolen-phone pattern would not be automatically dismissed. Logging their decisions creates new data to check later whether analysts use the field well. The team can revisit automation if more evidence accumulates. A review after three months will check whether analysts use the field well and whether any real takeover was wrongly set aside.

Sharing results

Amara presents the findings at the SOC's monthly review and stores the analysis script with the case data references.

Worked example

Amara's report concludes: "Device continuity is a useful signal but not a safe filter: in one of 12 real cases the attacker used the customer's own stolen device. We display it as context and will re-measure after three months."

Synthetic case. Figures are invented.

Errors and limits

Eighty-two alerts from one company give rough estimates; results may not transfer elsewhere.

Exercises

Synthesis

Applied research brings published ideas and careful local measurement together to make safer engineering decisions.

Continue studying — chapter 42 of 50

← Chapter 41 (draft) · Course chapter index · Chapter 43 (draft) →