OpenCyber Academy
Postgraduate · Chapter 5 · English
Course code: OCA-157Chapter 05: OCA-157-C005

Chapter 5 — Design alternatives for assessment programmes

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 05 published. Map, example, EX021.

flowchart TD
  N0["Learning outcomes"]
  N1["5.1 Alternatives"]
  N2["5.2 Scoring"]
  N3["5.3 Portfolio"]
  N4["5.4 Non-goals"]
  N5["5.5 Sensitivity"]
  N6["5.6 Output"]
  N7["5.7 Narrative"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX021"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["5.1 Alternatives"]
  N2["5.2 Scoring"]
  N3["5.3 Portfolio"]
  N4["5.4 Non-goals"]
  N5["5.5 Sensitivity"]
  N6["5.6 Output"]
  N7["5.7 Narrative"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX021"]

[Learning outcomes]
    |
    v
[5.1 Alternatives]
    |
    v
[5.2 Scoring]
    |
    v
[5.3 Portfolio]
    |
    v
[5.4 Non-goals]
    |
    v
[5.5 Sensitivity]
    |
    v
[5.6 Output]
    |
    v
[5.7 Narrative]
    |
    v
[Worked example] --> [Exercise EX021]

Learning outcomes

You will (1) generate programme alternatives; (2) score them on learning, risk, cost and capacity; (3) recommend a portfolio; (4) document non-goals.

5.1 Alternatives

Amira writes four alternatives so executives see real choices: A, quarterly web only; B, annual threat-led only; C, quarterly web plus biennial threat-led; and D, continuous automated scan theatre. She will not discuss a vague "more testing" request until it maps to one of these four, and a request that does not map is sent back until it does. DEC-14 is the record of choosing C after A, B and D were scored. Writing the alternatives side by side makes the rejected options as visible as the chosen one, which is what a later examiner needs.

5.2 Scoring

Amira scores each alternative on detection learning, operational risk, cost, staff capacity and examiner fit, each criterion from one to three. The raw numbers stay visible so a director can recompute the ranking by hand. D scores high on noise and low on learning. B rates well on learning but badly on staff capacity, because annual threat-led work needs two full white-team cycles. C wins for Meridian Clearing House when capacity is honest, and the score sheet attached to the alternatives memo keeps the recommendation from becoming a matter of taste.

5.3 Portfolio

C usually wins for MCH-like organisations because A alone under-trains detection and B alone misses ordinary regressions between threat-led years. Portfolio language means the mix of activities, not a single purchased logo or a single silver engagement. Amira still shows the scores for A, B and D, so directors understand why C won rather than taking the recommendation on trust. When a director later proposes dropping the quarterly web tests to fund a bigger threat-led year, the same scores show what detection learning and regression coverage that trade would cost.

5.4 Non-goals

The memo lists three non-goals: no claim of SWIFT coverage from ordinary web tests; no public exploit demos; no unpaid overtime as a capacity plan. Each non-goal maps to a catalogue cannot-claim line or to the ethics charter, so none of them is a new promise invented for the memo. Amira reads them aloud at the end of the alternatives memo so unpaid overtime cannot quietly become the capacity plan. Before filing, she checks every recommendation against the staffing rota the owners actually run; a recommendation that only works if people work nights is not executable.

5.5 Sensitivity

Amira re-scores the alternatives under shocks, because the recommendation should still be executable when conditions worsen. Her sensitivity analysis halves capacity and also doubles vendor cost. Doubling vendor cost hits option B hardest, since it buys a threat-led provider every year instead of every second year. Current C survives halved capacity by slipping threat-led to every third year with a named white-team lead and a written date. If C collapsed under both shocks, it was never robust; Amira refuses a plan that only works in the best case.

5.6 Output

The output is a two-page alternatives memo carrying the score table, the non-goals, the sensitivity result and a calendar graphic. The graphic shades the preparation months — intelligence gathering, white-team briefing and scenario sign-off, each in a different tone — and not only the execution week. Finance once funded only the execution week and was surprised by intelligence lead time; the shaded cells stopped that surprise. The memo's last line names the CISO as owner of the recommendation and the next catalogue review as its expiry. Amira will not send a memo that lacks the calendar graphic, because executives would then approve a programme they have not staffed.

5.7 Narrative

Amira reads the non-goals at the start of every alternatives workshop, so brainstorming cannot quietly reintroduce theatre option D, and again at the end, so nobody leaves thinking overtime will save the plan. A memo without sensitivity analysis is incomplete. When capacity halves, slipping threat-led to every third year keeps a named white-team lead and a written date rather than a wish. The slip itself is filed as a DEC amendment, so the cadence change is owned in the same way as the original choice of C.

5.8 Pencilled leads

Every-third-year threat-led still has a named white-team lead pencilled two years out; pencils beat wishes. Naming the lead that early gives the white team time to arrange cover before the exercise is due, and cover arranged late becomes the heroics the ethics charter forbids. Amira puts the name in the calendar graphic, not only in an email. The pencilled name is checked at each catalogue review, and any change of person is recorded as a DEC amendment. This is how Meridian Clearing House keeps every-third-year threat-led executable.

5.9 Shaded calendar

Each shaded preparation month in Amira's calendar graphic carries the initials of the white-team member pencilled for it, so executives see the true staff load and gaps in cover show up as blank cells. When a cell stays blank, Amira raises the gap with the white-team lead before the month begins rather than discovering it in the execution week. She reprints the calendar after every DEC that changes cadence and stores the PDF hash beside DEC-14, so the version a director approved can be told apart from later redrafts.

Worked example (synthetic MCH)

Amira's memo selects C, shows a two-year calendar, and lists D as rejected for false assurance.

Synthetic case — Meridian Clearing House (MCH). The clearing organisation, Amira Hassan and all figures are invented for teaching.

Errors and limits

Threat-led and clearing-system tests require specialised legal authorization. This book uses synthetic scenarios only and never supplies exploit code or live targeting advice for market infrastructures.

Exercises

Synthesis

Programme design is a scored portfolio with non-goals, not a single purchased logo.

Continue studying — chapter 05 of 12

← Chapter 04 (draft) · Course chapter index · Chapter 06 (draft) →

Worked answers

EX021. It creates noise and false assurance without detection learning.

EX022. It catches web regressions quarterly and still funds periodic threat-led learning.

EX023. No claim of SWIFT coverage from ordinary web tests.

EX024. A scenario where staff capacity halves (or similar).

EX025. A two-year calendar showing when each activity runs.