Public portal disruption
A public service portal slows sharply during a traffic surge; some legitimate users cannot complete submissions.
Fictional training scenario; no real system or person is described.
Safety and authority boundary
Do not block entire countries or providers without impact analysis; preserve a path for legitimate users.
Preparation
GV · ID · PRA public service portal slows sharply during a traffic surge; some legitimate users cannot complete submissions.
Your decision. Before opening the response, decide what each named team should do, how it would do it, why, whose approval is needed, and what evidence would change the decision.
Reveal the worked team response
| Team | What | How | Why |
|---|---|---|---|
| Pine Evidence team | Prepared an evidence route | Named read-only sources and custodians for: Edge metrics show requests from many networks, while application error rate rises only on the submission endpoint. | The first alert needs a verifiable provenance and time. |
| Quartz Service team | Rehearsed safe continuity | Tested an isolated version of: Scale the submission service and remove temporary controls once traffic and application behavior stabilize. | A rushed recovery can worsen the service impact. |
| River Decision team | Set authority and escalation | Assigned owners and approval conditions around: Do not block entire countries or providers without impact analysis; preserve a path for legitimate users. | Safety, privacy and communications need accountable decisions. |
Detection and analysis
DEEdge metrics show requests from many networks, while application error rate rises only on the submission endpoint.
Your decision. Before opening the response, decide what each named team should do, how it would do it, why, whose approval is needed, and what evidence would change the decision.
Reveal the worked team response
| Team | What | How | Why |
|---|---|---|---|
| Pine Evidence team | Triaged the signal | Preserved source, UTC time and a second independent check for: Edge metrics show requests from many networks, while application error rate rises only on the submission endpoint. | One alert is a lead, not proof of cause or scope. |
| Quartz Service team | Measured user impact | Compared the actual service with the scenario baseline: A public service portal slows sharply during a traffic surge; some legitimate users cannot complete submissions. | Technical alerts alone do not establish operational impact. |
| River Decision team | Declared a bounded case | Recorded confirmed facts, unknowns, owner and next review time. | The teams can coordinate without overstating the evidence. |
Response and containment
RSDo not block entire countries or providers without impact analysis; preserve a path for legitimate users.
Your decision. Before opening the response, decide what each named team should do, how it would do it, why, whose approval is needed, and what evidence would change the decision.
Reveal the worked team response
| Team | What | How | Why |
|---|---|---|---|
| Pine Evidence team | Preserved and scoped evidence | Saved the original records and defined a bounded path for: Apply bounded rate controls at the affected endpoint and compare legitimate-user completion before widening rules. | Containment must not destroy the basis for later review. |
| Quartz Service team | Applied the approved control | Implemented: Apply bounded rate controls at the affected endpoint and compare legitimate-user completion before widening rules. | A reversible, owner-approved change reduces harm without expanding scope. |
| River Decision team | Authorized and communicated | Logged the approver, affected users and constraint: Do not block entire countries or providers without impact analysis; preserve a path for legitimate users. | Response actions need clear authority and proportional notice. |
Recovery
RCScale the submission service and remove temporary controls once traffic and application behavior stabilize.
Your decision. Before opening the response, decide what each named team should do, how it would do it, why, whose approval is needed, and what evidence would change the decision.
Reveal the worked team response
| Team | What | How | Why |
|---|---|---|---|
| Pine Evidence team | Checked for persistence | Compared new telemetry against: Real users finish a test submission, error rates fall, and no rule disproportionately blocks normal clients. | An apparently healthy service can still be compromised. |
| Quartz Service team | Restored in stages | Executed: Scale the submission service and remove temporary controls once traffic and application behavior stabilize. | A staged return tests dependencies before widening access. |
| River Decision team | Accepted the business outcome | Requested owner sign-off only after: Real users finish a test submission, error rates fall, and no rule disproportionately blocks normal clients. | Recovery is a service decision, not merely a green dashboard. |
Improvement
ID.IMTest surge capacity and assess mitigation cost to users before an emergency.
Your decision. Before opening the response, decide what each named team should do, how it would do it, why, whose approval is needed, and what evidence would change the decision.
Reveal the worked team response
| Team | What | How | Why |
|---|---|---|---|
| Pine Evidence team | Turned uncertainty into a test | Documented the evidence gap and proposed: Test surge capacity and assess mitigation cost to users before an emergency. | A lesson needs a measurable future check. |
| Quartz Service team | Rehearsed the changed control | Assigned an owner and date to test: Test surge capacity and assess mitigation cost to users before an emergency. | An untested action item is not a verified improvement. |
| River Decision team | Closed the decision record | Approved resources, tracked the owner and checked the new control later. | Learning must feed Govern, Identify and Protect. |
Recovery acceptance test
Real users finish a test submission, error rates fall, and no rule disproportionately blocks normal clients.
Improvement to verify
Test surge capacity and assess mitigation cost to users before an emergency.