Chapter 34 — Comparative project
Lesson map
Lesson 34 published. Map, example, EX067.
flowchart TD
N0["Learning outcomes"]
N1["34.1 Criteria and weights"]
N2["34.2 Scoring and sensitivity"]
N3["34.3 The project"]
N4["34.5 Practical activity"]
N5["34.7 Presenting the comparison"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX067"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["34.1 Criteria and weights"]
N2["34.2 Scoring and sensitivity"]
N3["34.3 The project"]
N4["34.5 Practical activity"]
N5["34.7 Presenting the comparison"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX067"]
[Learning outcomes]
|
v
[34.1 Criteria and weights]
|
v
[34.2 Scoring and sensitivity]
|
v
[34.3 The project]
|
v
[34.5 Practical activity]
|
v
[34.7 Presenting the comparison]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX067]
Learning outcomes
You will (1) compare two or more migration approaches on explicit criteria; (2) weight criteria transparently; (3) test the sensitivity of the conclusion to weights and assumptions; and (4) present a defensible recommendation.
34.1 Criteria and weights
Comparisons need criteria agreed in advance: protection against HNDL, protection against forge-later, operational risk, cost, time to deploy, agility for future change and dependency on third parties. Weights express priorities; publish them so readers can disagree with the weights rather than with hidden judgments.
A criterion that cannot be scored with evidence should be reworded or dropped. For HCU, the weights were set by the risk owner after discussion with operations and finance, and each criterion has a short definition so that two scorers would read it the same way. HNDL protection carries the largest weight because of the mortgage archive.
34.2 Scoring and sensitivity
Score each alternative per criterion on a simple scale with evidence. Multiply by weights and sum. Then vary weights and key assumptions (CRQC timeline, vendor delivery dates) to see whether the ranking changes. A recommendation that holds across reasonable weightings is robust; one that flips with small changes should be presented as a close call.
Record each score with its source. A score with no evidence behind it is a guess and should be marked as one. Sensitivity runs are chosen where reviewers disagree most, not at random, so that the analysis answers the questions people are actually asking.
34.3 The project
Students compare, for HCU's branch WAN, three options: (A) hybrid IKEv2 via appliance firmware, (B) RFC 8784 post-quantum pre-shared keys as an interim layer, and (C) a new SD-WAN product with native hybrid support. Evidence comes from the system model (Chapter 21), lab measurements (Chapters 25 and 30) and vendor statements labelled as such.
Each option is described in enough detail that a reader can see what would actually change at a branch. The comparison covers all 40 branches, noting where a branch's appliance or link differs from the rest, because one exception can change which option is practical there.
34.4 Worked example
Synthetic scores (1–5) with weights HNDL 0.35, operational risk 0.2, cost 0.15, time 0.15, agility 0.15: A = 4,4,4,3,4 → 3.85; B = 3,3,5,5,2 → 3.45; C = 5,2,1,2,5 → 3.35. Sensitivity: if time weight rises to 0.3 (taking 0.15 from HNDL), B narrowly overtakes A (A 3.70 vs B 3.75). The recommendation becomes "A as target; B as interim at branches where firmware is delayed".
The weights were agreed with HCU's risk owner before scoring began, and the sensitivity run was chosen because delivery time was the factor most disputed in review. Branches waiting on firmware are named in the plan.
34.5 Practical activity
Complete the comparison with your own scores and evidence; run at least two sensitivity variations; write a one-page recommendation.
Where a score depends on a vendor statement, label it and say how you would check it. In the recommendation, state the option, the main reason, the sensitivity result that could change it, and the first action HCU should take. Keep the page readable by someone who will not open the scoring table. Then ask a classmate to change one weight and see whether your recommendation still holds.
34.6 Common errors and limits
- Choosing criteria after scoring.
- Treating vendor claims as measured evidence.
- Presenting a single ranking without sensitivity.
- Using too many criteria, so each carries little weight and the scoring becomes noise.
- Double-counting one concern under two names, such as cost and time when both measure the same vendor delay.
- Scores without recorded evidence.
- Omitting the option of doing nothing yet, which is a real choice with its own risks.
Review the criteria list with someone outside the project before scoring. Keep notes of that review.
34.7 Presenting the comparison
Show the scoring table, the weights and the sensitivity results on one page, followed by the evidence behind each score. Readers should be able to change one weight in their head and see whether the conclusion survives. Where a score rests on a vendor statement rather than measurement, mark it, because those scores are the first to revisit when the vendor delivers or slips.
Keep the version history of the table. When a vendor delivers late or early, update the affected score, rerun the sensitivity checks and note the date, so readers can see how and why the recommendation evolved.
Exercises
- EX067 — Recompute. Verify the weighted totals in 34.4 for A and B under the changed weights. Rubric: 3 = correct arithmetic shown.
- EX068 — Criterion. Propose an additional criterion relevant to HCU and explain how you would score it with evidence. Rubric: 3 = relevant, measurable criterion.
Synthesis
Transparent criteria, weights and sensitivity analysis turn a comparison of migration options into a recommendation that others can check and challenge.
Continue studying — chapter 34 of 40
← Chapter 33 (draft) · Course chapter index · Chapter 35 (draft) →