Chapter 34 — Comparative project | Bachelor's Critical infrastructure and sector security

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 34 published. Map, example, EX068.

flowchart TD
  N0["Learning outcomes and connection"]
  N1["34.1 Multi-criteria comparison"]
  N2["34.2 Doing it honestly"]
  N3["34.3 When a score is not enough"]
  N4["34.5 Project deliverable"]
  N5["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5
  N5 --> EX["Worked example"]
  EX --> Q["Exercise EX068"]

Logical drawing

flowchart LR
  N0["Learning outcomes and connection"]
  N1["34.1 Multi-criteria comparison"]
  N2["34.2 Doing it honestly"]
  N3["34.3 When a score is not enough"]
  N4["34.5 Project deliverable"]
  N5["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX068"]

[Learning outcomes and connection]
    |
    v
[34.1 Multi-criteria comparison]
    |
    v
[34.2 Doing it honestly]
    |
    v
[34.3 When a score is not enough]
    |
    v
[34.5 Project deliverable]
    |
    v
[Chapter synthesis]
    |
    v
[Worked example] --> [Exercise EX068]

Learning outcomes and connection

After this project you can (1) compare three or more options against explicit criteria with weights and evidence; (2) test how sensitive the ranking is to the weights; (3) present a comparison that a decision-maker can challenge; and (4) recognise when a weighted score hides a decisive consideration. It builds on Chapters 1, 15, 23 and 24 and prepares the capstone proposal (Chapter 35). It is the course's central expression of its outcome: compare methods, evaluate evidence and design defensible solutions.

34.1 Multi-criteria comparison

Decisions about critical services involve several criteria that do not share a unit: consequence reduction, operational burden, cost, time to implement, strength of evidence that the option works. Multi-criteria decision analysis scores each option on each criterion, weights the criteria, and combines them — most simply as a weighted sum. Its value lies less in the final number than in making the criteria, scores and weights visible so that people can argue about them productively.

In the unsupported-devices capstone evaluation (Chapter 39), the cost weight is printed beside each score, which is how a reader can see that the vendor gateway wins once that weight passes 0.45.

34.2 Doing it honestly

Define each criterion so that higher is better (invert costs and burdens). Score on an anchored scale ("5 = removes the worst credible consequence; 1 = no measurable effect"). Record the evidence behind each score (Chapter 15). Set weights before scoring, ideally with stakeholders, to avoid tuning weights to a favourite. Then run sensitivity analysis: vary the weights and see whether the winner changes. If it changes only under implausible weights, the decision is robust; if small changes flip it, report that the decision depends on a value judgement and name it.

34.3 When a score is not enough

Some considerations are constraints, not criteria: a legal obligation, a safety requirement, a minimum service level. An option that violates a constraint is excluded, whatever its score. Others are vetoes by people with authority (a clinical director's refusal of an option that disrupts theatres). Handle these explicitly before scoring.

For CEDC, a regulator's four-hour restoration requirement is a constraint (EX068), excluding options that cannot demonstrate it.

A clinical director's veto of an option that closes theatres for a week is handled the same way.

34.4 Worked example and Lab 13 (synthetic)

This case is synthetic. The CRP compares three ransomware-resilience options for CEDC on four criteria with weights 0.4, 0.2, 0.2, 0.2.

# Lab 13 - weighted comparison of three options and weight sensitivity (synthetic scores, offline)
CRITERIA = ["consequence reduction", "operational burden (inverted)", "cost (inverted)", "evidence strength"]
W = [0.4, 0.2, 0.2, 0.2]
SCORES = {  # 1-5, higher is better
    "network segmentation project": [5, 2, 2, 4],
    "managed detection service":    [3, 4, 3, 3],
    "offline backups + restore drills": [4, 4, 4, 5],
}
def total(s, w): return sum(a * b for a, b in zip(s, w))
for name, s in SCORES.items():
    print(f"{name:33} {total(s, W):.2f}")
print("weight on consequence needed for segmentation to win:")
for wc in (0.4, 0.5, 0.6, 0.7, 0.8):
    rest = (1 - wc) / 3
    w = [wc, rest, rest, rest]
    best = max(SCORES, key=lambda k: total(SCORES[k], w))
    print(f"  w_consequence={wc:.1f} -> {best}")

Expected output:

network segmentation project      3.60
managed detection service         3.20
offline backups + restore drills  4.20
weight on consequence needed for segmentation to win:
  w_consequence=0.4 -> offline backups + restore drills
  w_consequence=0.5 -> offline backups + restore drills
  w_consequence=0.6 -> offline backups + restore drills
  w_consequence=0.7 -> network segmentation project
  w_consequence=0.8 -> network segmentation project

Backups with restore drills win under the agreed weights and remain ahead until consequence reduction carries between 0.6 and 0.7 of the total weight. The report therefore says: "Recommended: offline backups with restore drills. The recommendation reverses only if the partnership values consequence reduction at more than about 0.63 of the decision (EX067 derives the exact crossover), in which case segmentation is preferred. The options are not exclusive; segmentation is the strongest second-year candidate." Note what the scores do not capture: backups recover from ransomware but do not prevent the data theft that often accompanies it — a consideration to state explicitly, not hide in a number.

34.5 Project deliverable

A 2,500-word comparative report on a question of your choice within the Calder case or a documented public topic (for example, three approaches to remote access, three backup strategies, three monitoring options): constraints, criteria with anchored scales, evidence per score, weights set in advance, sensitivity analysis, recommendation with reversal conditions, and limits.

For the unsupported-devices capstone comparison (Chapter 39) that weight is 0.45 on cost; below it isolation with monitoring wins, above it the vendor gateway does.

State the reversal condition in words, as Lab 13's report does: the weight at which the recommendation would change and the value judgement that weight represents.

34.6 Common errors and limits

Scoring first and choosing weights afterwards.
Unanchored scores ("4 because it feels good").
Ignoring constraints and vetoes.
Presenting a weighted total as objective truth.
Treating options as mutually exclusive when they can be combined.

Lab 13 avoids the first by fixing weights at 0.4, 0.2, 0.2, 0.2 before scoring. The report also notes that segmentation and backups can be combined, avoiding the fifth error. A score of 4 for evidence strength needs the drill records behind it; without them it is the second error.

Presenting 4.20 as proof would be the fourth.

Exercises — chapter 34

EX067 — Find the crossover. Objective: locate the reversal point. Inputs: Lab 13. Steps: compute the exact consequence weight at which segmentation and backups tie, using the equal-split rule for the other weights. Expected evidence: derivation and value. Rubric: 3 = segmentation $5w + (2+2+4)(1-w)/3$, backups $4w + (4+4+5)(1-w)/3$; tie when $w = 5(1-w)/3$, so $w = 5/8 = 0.625$; 2 = numeric search to 0.62–0.63; 1 = range 0.6–0.7 only; 0 = none.

EX068 — Add a constraint. Objective: handle a non-compensatory requirement. Inputs: CEDC's regulator requires demonstrated restoration within 4 hours. Steps: explain how this changes the analysis and which evidence each option must provide. Expected evidence: explanation. Rubric: 3 = treated as constraint not criterion; only options that demonstrate restoration qualify (backups with drills directly; segmentation/detection do not restore, so need to be combined with a recovery capability); 2 = treated as heavy weight; 1 = ignored; 0 = none.

Chapter synthesis

Comparison is the bachelor's core skill: explicit criteria, anchored scores, evidence, weights set in advance, sensitivity analysis and stated reversal conditions. Constraints and vetoes come before scoring, and a weighted total is a structured argument, not a verdict.