Chapter 11 — Supervised laboratory

Chapter 11 — Supervised laboratory

Lesson map

Lesson 11 published. Map, example, EX051.

flowchart TD
  N0["Learning outcomes and connection"]
  N1["11.1 Types of exercise"]
  N2["11.2 Design"]
  N3["11.3 Facilitation"]
  N4["11.4 Evaluation and improvement"]
  N5["11.6 Exercise-design rubric"]
  N6["11.7 Practical activity supervised"]
  N7["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX051"]

Logical drawing

flowchart LR
  N0["Learning outcomes and connection"]
  N1["11.1 Types of exercise"]
  N2["11.2 Design"]
  N3["11.3 Facilitation"]
  N4["11.4 Evaluation and improvement"]
  N5["11.6 Exercise-design rubric"]
  N6["11.7 Practical activity supervised"]
  N7["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX051"]

[Learning outcomes and connection]
    |
    v
[11.1 Types of exercise]
    |
    v
[11.2 Design]
    |
    v
[11.3 Facilitation]
    |
    v
[11.4 Evaluation and improvement]
    |
    v
[11.6 Exercise-design rubric]
    |
    v
[11.7 Practical activity supervised]
    |
    v
[Chapter synthesis]
    |
    v
[Worked example] --> [Exercise EX051]

Learning outcomes and connection

After this chapter you should be able to:

• design a discussion-based tabletop exercise for an essential service, with objectives, scenario, injects, roles and evaluation criteria;
• facilitate it safely and fairly, keeping it a test of plans rather than of people;
• evaluate it with a hot wash, an after-action report and an improvement plan;
• assess an exercise design against a rubric under supervision.

For a professional coordinator, the laboratory is the exercise room. This chapter assembles everything so far — stakeholders, authorization, risk, RACI, supplier access, notification clocks, decision quality — into one supervised exercise.

11.1 Types of exercise

Exercise programmes distinguish discussion-based exercises (seminars, workshops, tabletops), where participants talk through a scenario, from operations-based exercises (drills, functional exercises, full-scale exercises), where they carry out actions. The US Homeland Security Exercise and Evaluation Program (HSEEP) sets out this progression and a design–conduct–evaluate–improve cycle (FEMA, HSEEP). NIST's guide covers test, training and exercise programmes for IT plans (NIST SP 800-84). CISA publishes ready-made tabletop exercise packages for critical-infrastructure scenarios (CISA Tabletop Exercise Packages), and the UK NCSC offers "Exercise in a Box" (NCSC, Exercise in a Box).

A tabletop is the right first step: cheap, safe — nothing touches a live system — and excellent at exposing gaps in roles, information and communication.

11.2 Design

A design document has seven parts:

1. Objectives — three to five, each observable: "Within 30 minutes of the relevant inject, the team decides whether to suspend supplier access, with owner and rationale recorded." Avoid "test our response".
2. Scope and assumptions — which plans are tested, which teams play, what is out of scope ("the technical forensic process is not played").
3. Scenario — plausible, synthetic, sector-appropriate, based on public threat information at a general level; never a real colleague's mistake.
4. Injects — timed pieces of new information (an alarm, a phone call, a journalist, a regulator deadline). Each inject maps to at least one objective and has an expected decision.
5. Roles — players (decide as they would), facilitator (runs the clock, presents injects, keeps discussion moving), evaluators (observe against objectives, keep the decision log), scribe, and a safety officer or trusted agent who can pause the exercise.
6. Evaluation criteria — what "met", "partially met" and "not met" mean for each objective.
7. Logistics and rules — date, room, no real systems touched, "exercise, exercise, exercise" on every written inject so nothing can be mistaken for a real event, and a real-world stop phrase.

11.3 Facilitation

Open with ground rules: this tests plans, not people; there are no trick questions; decide as you would in reality; say when you would need information you do not have. Keep the clock honest and resist rescuing players — if the team has no way to reach the hospital, the gap is the finding. Record decisions, not opinions about performance.

In DRY CREEK the facilitator did not supply the missing hospital contact or assign the call when players hesitated; that silence produced the clearest finding of the exercise.

11.4 Evaluation and improvement

Immediately after: a hot wash — each participant says what worked and what did not. Within two weeks: an after-action report (AAR) that rates each objective with evidence from the decision log, and an improvement plan that turns findings into owned, dated, verifiable actions (Chapter 10). The AAR is shared at the appropriate TLP level; it names gaps in plans and tools, not individuals.

The DRY CREEK report rated O3 as not met and turned it into one action with an owner, a date and a re-test inject.

11.5 Worked example: exercise "DRY CREEK" (synthetic)

Objectives. (O1) Declare or decline a cyber incident within 15 minutes of the first indicator, with rationale. (O2) Decide on suspending a supplier's remote access within 30 minutes, knowing which operations depend on it. (O3) Brief the hospital within its 2-hour agreement, with one named lead. (O4) Meet the illustrative regulator early-warning deadline with an approved message. (O5) Issue a holding statement with an accountable approver within 45 minutes of media contact.

Scenario. On a Friday evening, a HydroLogic engineer's laptop is reported stolen; an hour later, KVROC reports a remote session to station KW-04 from HydroLogic's account at an unusual hour, and an operator sees an unexpected change of dosing setpoint, which the controller's limits reject. All synthetic.

Inject table (extract).

Roles. Players: E. Varga (executive), A. Mensah, L. Ferreira, R. Adeyemi, the on-call engineer, M. Holt joining at 00:55. Facilitator: the coordinator. Evaluators: an internal auditor and a peer coordinator from a neighbouring utility under a confidentiality agreement. Safety officer: shift supervisor, with the stop phrase "real world, real world".

Evaluation (from the evaluator's log, analysed with the Lab 10 method). O1 met (9 minutes, rationale recorded). O2 partially met — decision at 52 minutes because nobody knew who else relied on HydroLogic's access. O3 not met — the hospital call was never assigned. O4 met. O5 partially met — statement issued late, no owner until the chief executive arrived. These findings fed directly into the corrective actions of Chapter 10.

11.6 Exercise-design rubric

Use the rubric on the DRY CREEK design: its objectives are observable and time-bound, each inject maps to one.

11.7 Practical activity (supervised)

Design a 90-minute tabletop for an essential service of your choice using the seven parts of 11.2, with at least four objectives and six injects. Ask your supervisor or a peer to score it against the rubric before you run it.

Base the scenario on public threat information at a general level, as DRY CREEK used a stolen supplier laptop and an unusual remote session, and never on a real colleague's mistake. Map every inject to an objective with an expected decision, and name a safety officer with a stop phrase. Mark every written inject as an exercise. After scoring, revise the design and record what the reviewer changed, so the supervisor can see how feedback was used.

11.8 Common errors and limits

• Too many objectives; nothing is evaluated properly.
• Scenarios so severe that every plan fails and nothing specific is learned.
• Facilitators who rescue players or let senior people dominate.
• Exercise injects that could be mistaken for real messages.
• AARs that never become actions.

• A tabletop shows whether plans and roles make sense; it does not show that procedures can be carried out at speed — that needs drills.

• Choosing evaluators who are also players, so nobody observes the decisions independently.

• Ignoring a hot-wash remark such as "we assumed comms had it", which in DRY CREEK pointed to the real cause.

Exercises — chapter 11

EX051 — Make objectives observable. Objective: write evaluable objectives. Inputs: "Test our communication", "Check supplier management", "See if people know the plan". Steps: rewrite each. Expected evidence: three objectives. Rubric: 3 = each states a decision or action, a time limit and what is recorded; 2 = no time limits; 1 = still vague; 0 = none.

EX052 — Write injects. Objective: map injects to objectives. Inputs: DRY CREEK objectives. Steps: write two new injects that test O2 and O4 more deeply, with expected decisions. Expected evidence: two inject rows. Rubric: 3 = plausible, synthetic, each mapped with expected decision (for example HydroLogic asks to reconnect to fix dosing; regulator duty officer asks for an update before the deadline); marked "exercise"; 2 = unmapped; 1 = unrealistic; 0 = none.

EX053 — Score a design. Objective: apply the rubric. Inputs: a colleague's design: one objective "test incident response"; scenario is a nation-state attack causing a dam failure; the CEO evaluates; injects sent from the CEO's real email account. Steps: score each criterion and propose fixes. Expected evidence: scores and fixes. Rubric: 3 = low scores on objectives, scenario proportion, roles/evaluation independence and safety (real email risks confusion); fixes for each; 2 = fixes without scores; 1 = accepts; 0 = none.

EX054 — Write the AAR summary. Objective: evaluate against objectives. Inputs: DRY CREEK evaluation in 11.5. Steps: write the AAR summary table with rating, evidence and recommended action per objective. Expected evidence: five rows. Rubric: 3 = ratings consistent with evidence, actions owned and verifiable, no individual blamed; 2 = actions vague; 1 = narrative; 0 = none.

EX055 — Facilitation challenge. Objective: handle a facilitation problem. Inputs: in the exercise, the chief executive dominates and decides everything, including technical isolation. Steps: describe what the facilitator does during and after. Expected evidence: actions. Rubric: 3 = during: invite role owners by name, ask "who would decide this in reality?", record; after: raise in AAR as a plan/role clarity finding, brief the executive privately and respectfully; 2 = only after; 1 = ignores; 0 = none.

Chapter synthesis

A tabletop exercise is the coordinator's laboratory: safe, cheap and powerful at exposing gaps in roles, information and communication. Design it with observable objectives, a plausible synthetic scenario, injects mapped to objectives, independent evaluators and a safety officer. Facilitate to test plans, not people; evaluate with a hot wash, an after-action report and an improvement plan whose actions are owned and verified.

Continue studying — chapter 11 of 12

← Chapter 10 · Course chapter index · Chapter 12 →

Continue to the next authored lesson.