Chapter 35 — Design studio II | Engineering Security operations and blue team

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 35 — Design studio II

Lesson map

Lesson 35 published. Map, example, EX103.

flowchart TD
  N0["Learning outcomes"]
  N1["35.1 The problem"]
  N2["35.2 Options explored"]
  N3["35.3 Analysis"]
  N4["35.4 Human factors"]
  N5["35.5 The design"]
  N6["35.6 Critique of others"]
  N7["35.7 Readiness"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX103"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["35.1 The problem"]
  N2["35.2 Options explored"]
  N3["35.3 Analysis"]
  N4["35.4 Human factors"]
  N5["35.5 The design"]
  N6["35.6 Critique of others"]
  N7["35.7 Readiness"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX103"]

[Learning outcomes]
    |
    v
[35.1 The problem]
    |
    v
[35.2 Options explored]
    |
    v
[35.3 Analysis]
    |
    v
[35.4 Human factors]
    |
    v
[35.5 The design]
    |
    v
[35.6 Critique of others]
    |
    v
[35.7 Readiness]
    |
    v
[Worked example] --> [Exercise EX103]

Learning outcomes

You will (1) design a solution to a cross-cutting SOC problem in a team; (2) balance technical, human and cost factors; (3) produce a design ready for implementation; (4) critique other teams' designs constructively.

35.1 The problem

Night shift has two analysts. Studio II asks teams to reduce night alert load by 40% without increasing missed priority incidents.

Two analysts at night cannot investigate everything that arrives, so some alerts wait or are closed quickly. Reducing load without increasing missed incidents is a constraint pulling in two directions, which is what makes it a design problem rather than a simple cut. Teams must show evidence for both halves of the goal. Teams receive a month of anonymised alert data and the incident list for the same period, so their proposals can be checked against what actually happened.

35.2 Options explored

Amara's team considers: moving low-severity rules to a morning queue; a managed provider for overflow; automation of enrichment so each alert takes less time; and stricter rule promotion criteria.

Each option attacks load differently: rescheduling removes alerts from the night entirely, overflow shares them with outsiders, enrichment makes each one quicker, and promotion criteria stop new noisy rules from reaching nights. They can be combined, but each has risks, such as morning backlogs or the provider's lack of context. Costs are estimated for each.

35.3 Analysis

Using a month of alert data, they find 55% of night alerts come from five rules, three of which have never led to an incident. Moving those three to morning review cuts night load by 31%. Automated enrichment saves about two minutes per remaining alert.

Data change the discussion from opinion to evidence. Five rules generating over half the night alerts is a typical concentration, and three of them with no incidents ever is strong evidence of low value at night. Two minutes saved on each remaining alert adds up across a shift, giving analysts time for real investigations.

35.4 Human factors

Analysts in the studio session warn that moving alerts to morning creates a morning backlog. The team schedules a 30-minute morning review slot and limits the queue.

Engineers might count the problem solved once alerts leave the night queue, but analysts know someone must handle them later. A morning backlog left unchecked would grow and push other work aside. Scheduling a fixed slot and limiting the queue keeps the moved work visible and bounded. The analysts also ask that morning review be shared on a rota, so it does not fall on whoever arrives first.

35.5 The design

Move three rules to morning review; automate enrichment for the top ten alert types; set promotion criteria for new night-paging rules (precision above 0.3 in monitor-only mode); review after one month.

Each element addresses a finding: rescheduling the three no-incident rules, enrichment for the most common alert types, and promotion criteria so new rules prove their value before paging anyone at night. A precision above 0.3 means at least three in ten alerts are real. The one-month review checks both load and missed incidents.

35.6 Critique of others

Amara's team questions another team's proposal to page only on "critical" assets: the asset inventory marks only 40% of systems, so many unclassified assets would never page.

The flaw is in the data, not the idea: paging only on critical assets would work if criticality were known for every system. With only 40% classified, the other 60% would never page, whatever happened on them. Amara's team suggests treating unclassified assets as critical until the inventory is completed. The other team accepts the point and revises its proposal before the final presentation.

35.7 Readiness

The design includes owners, a timeline and a rollback: rules can be restored to night paging in minutes.

Owners make sure each change has someone responsible for it. The timeline sets when each part goes live and when the review happens. Rollback in minutes means that if missed incidents rise, the three rules return to night paging immediately, which makes the trial safe to run. The one-month review compares night alert counts and missed priority incidents with the month before, using the same definitions. Paolo and the night-shift lead attend it together.

Worked example

The estimated effect: 31% fewer night alerts from rule moves plus time saved by enrichment, equivalent to a 42% reduction in analyst minutes. Risk: a real incident from a moved rule waits until morning; mitigated because those three rules have high-severity companions that still page.

Synthetic case. Data and designs are invented.

Errors and limits

Estimates from one month may not hold during seasonal peaks such as holiday deliveries.

Exercises

EX103 — Load source. What did the alert data reveal? Rubric: 3 = 55% from five rules; three never led to incidents; 2 = partial; 1 = vague; 0 = none.
EX104 — Morning backlog. How did the design address the analysts' warning? Rubric: 3 = scheduled 30-minute morning slot and queue limit; 2 = partial; 1 = ignored; 0 = none.
EX105 — Constructive critique. What weakness did Amara's team find in another design? Rubric: 3 = paging only critical assets misses unclassified 60%; 2 = partial; 1 = vague; 0 = none.

Synthesis

Cross-cutting studio problems need data, human input and constructive critique to reach implementable designs.

Continue studying — chapter 35 of 50

← Chapter 34 (draft) · Course chapter index · Chapter 36 (draft) →