Lesson map
Lesson 05 published. Map, example, EX021.
flowchart TD
N0["Learning outcomes"]
N1["5.1 The design question"]
N2["5.2 Alternatives"]
N3["5.3 Criteria"]
N4["5.4 Scoring"]
N5["5.5 Sensitivity"]
N6["5.6 Risks of C"]
N7["5.7 Presenting"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX021"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["5.1 The design question"]
N2["5.2 Alternatives"]
N3["5.3 Criteria"]
N4["5.4 Scoring"]
N5["5.5 Sensitivity"]
N6["5.6 Risks of C"]
N7["5.7 Presenting"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX021"]
[Learning outcomes]
|
v
[5.1 The design question]
|
v
[5.2 Alternatives]
|
v
[5.3 Criteria]
|
v
[5.4 Scoring]
|
v
[5.5 Sensitivity]
|
v
[5.6 Risks of C]
|
v
[5.7 Presenting]
|
v
[Worked example] --> [Exercise EX021]
Learning outcomes
You will (1) generate alternative designs for SOC coverage; (2) evaluate in-house, managed and hybrid options; (3) use weighted criteria and sensitivity checks; (4) present a recommendation with trade-offs.
5.1 The design question
Meridian's SOC is staffed 07:00–19:00; nights rely on on-call engineers. The contractor incident at 23:40 was caught only because the on-call engineer was awake. The board asks for 24/7 coverage options. The question therefore is not whether to cover nights but how. Daniel frames it as a choice among options, each scored against criteria agreed with the CISO, so the board can see why one is recommended and what it would cost. He sets the decision and its success measures down before scoring, so that the criteria cannot be adjusted after the preferred option is known.
5.2 Alternatives
Option A: hire for in-house 24/7 (five more analysts). Option B: managed detection and response for nights. Option C: hybrid — managed tier-1 at night with in-house escalation and an automated containment playbook for high-confidence alerts. Each option answers the contractor incident differently. A gives Meridian analysts at 23:40 every night; B gives a provider who may not know which contractors run batch jobs; C gives a provider for first triage, Meridian's own engineers for decisions, and automation that can act within seconds on alerts above a confidence threshold. The automated playbook is limited to suspending export rights, not whole accounts.
5.3 Criteria
Detection quality, response speed, cost, local knowledge, data control and resilience. Each is weighted with the CISO. Weighting with the CISO makes the trade-offs explicit. Data control matters more at a health insurer than at many firms, because policyholder records must stay within agreed processing; local knowledge matters because many alerts depend on knowing which contractors work which hours. Daniel records the agreed weights in the board paper before any option is scored, so nobody can say the weights were chosen to favour a result.
5.4 Scoring
A scores high on knowledge and control but high cost and hiring risk; B is cheapest but weakest on knowledge; C balances them. Each score has a written reason in the board paper: A's hiring risk comes from the difficulty of recruiting five analysts for nights, and B's weakness on knowledge comes from the provider not knowing Meridian's contractors and claims system. The reasons let the board challenge a score instead of the whole recommendation. A balanced score for C reflects that its weaknesses can be managed with quality reviews and automation limits.
5.5 Sensitivity
Daniel checks whether the recommendation changes if weights shift. C stays best unless cost weight is doubled, when B wins narrowly. The recommendation is therefore robust to reasonable disagreement. The sensitivity check answers the board's obvious question: what if you weighted cost differently? Showing that C survives most plausible changes, and naming the one change under which B wins, makes the recommendation honest about where it is fragile. If the board cares mostly about cost, it can choose B knowing the trade-off it is accepting in local knowledge and escalation quality.
5.6 Risks of C
Managed analysts may escalate too much or too little; automation could wrongly disable accounts. Mitigations: quality reviews, escalation targets, automation limits. Daniel's success measures for C turn these risks into numbers the board can check after six months: escalation precision above 50%, and no wrongful automated containments of critical accounts. The automation is limited to suspending export rights, not disabling accounts, so a wrong trigger disrupts less. Quality reviews sample the provider's closed alerts each month. The board paper names an owner for each mitigation, such as the quality reviews of the provider's closed alerts.
5.7 Presenting
The board paper shows options, scores, sensitivity and risks on two pages. Two pages force Daniel to choose what matters: the board sees each option's score, how the recommendation holds when weights change, and the risks of C with mitigations. Supporting detail, such as the scoring reasons, sits in an appendix for members who want it. The board chose C with a six-month review, and the paper's success measures became the review's criteria. The appendix also lists the scoring reasons and the sensitivity table.
Stakeholder views
Daniel interviews analysts, the on-call engineers and the CISO before scoring. Analysts worry that a managed provider will send vague escalations at 03:00; engineers want fewer night calls. These views shape the escalation targets written into option C and make the final design one that the people running it can support.
Worked example
The board chooses C with a six-month review. Daniel defines success: night P1 alerts acknowledged within 15 minutes, escalation precision above 50%, no wrongful automated containments of critical accounts.
Synthetic case. Options and scores are invented.
Errors and limits
Scoring tables give an appearance of precision; the discussion around weights matters more than the numbers.
Exercises
- EX021 — Three options. Describe options A, B and C.
Show marking criteria
Rubric: 3 = in-house 24/7, MDR nights, hybrid with escalation and automation; 2 = two; 1 = one; 0 = none. - EX022 — Criteria. List the six evaluation criteria.
Show marking criteria
Rubric: 3 = detection, speed, cost, knowledge, control, resilience; 2 = four; 1 = two; 0 = none. - EX023 — Sensitivity. What did the sensitivity check show?
Show marking criteria
Rubric: 3 = C robust unless cost weight doubled (then B); 2 = partial; 1 = vague; 0 = none. - EX024 — Risks of C. Name two risks of option C and their mitigations.
Show marking criteria
Rubric: 3 = over/under escalation (reviews, targets); wrongful automation (limits); 2 = one; 1 = vague; 0 = none. - EX025 — Success measures. What success measures were set?
Show marking criteria
Rubric: 3 = night P1 ack ≤15 min, escalation precision >50%, no wrongful critical containments; 2 = two; 1 = one; 0 = none.
Synthesis
Generate real alternatives, score them on agreed criteria, test sensitivity and present trade-offs honestly.
Continue studying — chapter 05 of 12
← Chapter 04 (draft) · Course chapter index · Chapter 06 (draft) →