Chapter 34 — Comparative project: response strategy evaluation | Bachelor's Incident response and cyber resilience

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 34 published. Map, example, EX100.

flowchart TD
  N0["Learning outcomes and connection"]
  N1["34.1 Criteria"]
  N2["34.2 Scenario selection"]
  N3["34.3 Evidence strength"]
  N4["34.4 Trade-offs and sensitivity"]
  N5["34.5 Counterexamples"]
  N6["34.7 Report structure"]
  N7["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX100"]

Logical drawing

flowchart LR
  N0["Learning outcomes and connection"]
  N1["34.1 Criteria"]
  N2["34.2 Scenario selection"]
  N3["34.3 Evidence strength"]
  N4["34.4 Trade-offs and sensitivity"]
  N5["34.5 Counterexamples"]
  N6["34.7 Report structure"]
  N7["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX100"]

[Learning outcomes and connection]
    |
    v
[34.1 Criteria]
    |
    v
[34.2 Scenario selection]
    |
    v
[34.3 Evidence strength]
    |
    v
[34.4 Trade-offs and sensitivity]
    |
    v
[34.5 Counterexamples]
    |
    v
[34.7 Report structure]
    |
    v
[Chapter synthesis]
    |
    v
[Worked example] --> [Exercise EX100]

Learning outcomes and connection

You will be able to (1) define decision criteria and weights for comparing response strategies; (2) select scenarios that stress the differences; (3) grade the strength of evidence behind each judgement; (4) analyse trade-offs and sensitivity; and (5) write a comparative report with methods, counterexamples and limitations. This project draws on Chapters 18, 23, 24 and 33, and prepares the capstone sequence beginning in Chapter 35.

34.1 Criteria

Choose criteria that matter to the organization and can be assessed: time to containment, risk of service disruption, evidence preserved, number of handoffs, privacy intrusion, reliance on scarce skills. Define each with a scale and a method (simulation result, expert judgement, document review). Weights express priorities; agree them with stakeholders before scoring.

In the booking-platform comparison, the criteria are time to contain, service disruption, evidence preserved and handoffs, weighted at 35, 30, 15 and 20 per cent. Disruption scores were judged with the service owners, and time to contain was measured in a tabletop. The weights were agreed before any scoring, so the near-tie between the two playbooks cannot be blamed on weights chosen to produce it.

34.2 Scenario selection

A single scenario favours whichever strategy fits it. Use a small set that varies the factors that differentiate strategies: a fast-moving ransomware case, a slow credential-abuse case, a vendor-hosted incident, an incident during a critical clinical period.

The three booking-platform scenarios were chosen to pull the strategies apart: S1, phishing with immediate export, rewards speed; S2, a false alert at Monday clinic opening, punishes automatic disablement; S3, a shared front-desk account used by five people, makes disablement disruptive in a different way. A set of scenarios that all resembled S1 would have made P-Fast look clearly superior, and the weakness revealed by S2 would never have appeared.

34.3 Evidence strength

Grade each score: measured (from a simulation or data), documented (from a written procedure, not tested), judged (expert opinion). A matrix full of judged scores should be presented as an informed opinion, not a measurement.

In the booking-platform table, time to contain and handoffs are measured, evidence preservation is documented and disruption is judged with owners. Because the decisive weakness of P-Fast lies in a judged score, the recommendation should say that further testing of S2 would strengthen it. Carry the grade into the conclusion, not only into the table.

34.4 Trade-offs and sensitivity

Weighted totals hide trade-offs. Show the profile per criterion, then vary weights within plausible ranges to see whether the ranking changes (Chapter 24's sensitivity idea).

In the booking-platform comparison, the totals differ by less than a tenth of a point, yet the profiles differ sharply: P-Fast excels in S1 and fails in S2. Raising the disruption weight by ten points, taken from time, makes P-Gated lead, as the exercise on recomputation confirms. A ranking that flips under a modest, plausible change in weights is not a basis for a firm recommendation. Show the profile beside the totals so a reader sees why.

34.5 Counterexamples

For the recommended strategy, construct a scenario where it performs worse. If you cannot, you probably have not stressed it enough. Report the counterexample and how the organization would manage it.

For P-Fast, the counterexample is S2: a false high-risk alert at Monday clinic opening disables staff accounts just as patients arrive. That is what led to the hybrid recommendation of automatic session revocation, which is reversible and low in disruption, with gated disablement. For P-Gated, a fast automated exfiltration during a night shift with no approver available would be the counterexample.

34.6 Worked example (synthetic): two playbooks, one service

Fictional. Strategies for credential compromise of booking-platform staff accounts: P-Fast (automatic session revocation and account disablement on high-risk alert, review afterwards) and P-Gated (analyst confirms within 15 minutes, then incident lead approves containment).

Scenarios: S1 phishing with immediate data export; S2 false-positive risk alert during Monday morning clinic opening; S3 compromise of a shared front-desk account used by five people.

Scores (1–5, higher is better) with evidence grade: | Criterion (weight) | P-Fast S1/S2/S3 | P-Gated S1/S2/S3 | Evidence | |---|---|---|---| | Time to contain (0.35) | 5/5/5 | 3/3/3 | measured (tabletop) | | Service disruption (0.30) | 4/1/2 | 4/5/4 | judged with owners | | Evidence preserved (0.15) | 3/3/3 | 4/4/4 | documented | | Handoffs (0.20) | 4/4/3 | 3/3/3 | measured |

Averaged over scenarios, weighted totals: P-Fast = 0.35×5 + 0.30×(7/3) + 0.15×3 + 0.20×(11/3) ≈ 1.75 + 0.70 + 0.45 + 0.73 = 3.63; P-Gated = 0.35×3 + 0.30×(13/3) + 0.15×4 + 0.20×3 = 1.05 + 1.30 + 0.60 + 0.60 = 3.55. Nearly tied — but the profiles differ sharply: P-Fast is far better in S1 and far worse in S2 (disabling staff at clinic opening). Sensitivity: if disruption weight rises from 0.30 to 0.40 (taking 0.10 from time), P-Gated leads. Counterexample to P-Fast: S2. Recommendation: hybrid — automatic session revocation (low disruption, reversible) plus gated disablement — then re-test on all scenarios.

34.7 Report structure

Question and criteria; scenarios and why; methods for each score with evidence grade; results profile and totals; sensitivity; counterexamples; recommendation; limitations (judged scores, simulation realism, small scenario set).

For the booking-platform comparison, the methods part would explain that time to contain and handoffs came from a tabletop while disruption was judged with owners. The results part shows the per-scenario profile before the near-tied totals, and the counterexample part explains S2. Limitations name the three-scenario set, equal weighting of scenarios and possible bias in owners' judgements. Write limitations last but read them first when reviewing.

34.8 Common errors and limits

Weights chosen after scoring.
Single-scenario evaluation.
Presenting judged scores as measurements.
Recommending on totals while ignoring a catastrophic weak spot.
Treating a near-tie in totals as evidence that the strategies are equivalent, when their profiles differ sharply.
Choosing scenarios after scoring so that the preferred strategy wins.
Recommending a hybrid without re-testing it on all scenarios.
Letting one stakeholder group score the criterion that most affects them without a second view.
Hiding judged scores inside a weighted total so that a reader cannot tell which parts were measured.

Exercises — chapter 34

Attempt each exercise before opening the worked results. Every case, organization, person, address and number in an exercise is synthetic unless a cited public source is named.

EX100 — Recompute. Steps: recompute both totals with weights time 0.25, disruption 0.40, evidence 0.15, handoffs 0.20. Rubric: 3 = P-Fast ≈ 3.37, P-Gated ≈ 3.68, conclusion that ranking flips; 2 = arithmetic slip; 1 = not recomputed; 0 = none.

EX101 — Counterexample for P-Gated. Steps: describe a scenario where P-Gated performs badly and how to mitigate it. Rubric: 3 = fast automated exfiltration or night shift without available approver; mitigation such as pre-approved automatic session revocation or default-to-contain timer; 2 = scenario without mitigation; 1 = vague; 0 = none.

EX102 — Limitations section. Steps: write the limitations paragraph for the §34.6 report. Rubric: 3 = judged disruption scores, tabletop realism, three scenarios only, equal scenario weighting, owners' possible bias; direction of bias stated where possible; 2 = list without direction; 1 = generic; 0 = none.

Chapter synthesis

Compare strategies on pre-agreed criteria across several scenarios, grade the evidence behind every score, show profiles not just totals, test sensitivity, and always report the counterexample.