Chapter 03 — Containment, eradication and recovery decisions

Chapter 03 — Containment, eradication and recovery decisions

Five daily lessons · week 3 of four. This chapter groups the original section identifiers 3.x. Lesson numbers identify a learner day; section identifiers remain stable so existing source references and laboratory instructions still work.

Module map

Module 3 overview: five daily lessons below provide the teaching, examples, exercises and worked-answer links.

flowchart TD
  N0["Learning outcomes"]
  N1["3.1 A decision loop rather than a fixed sequence"]
  N2["3.2 The case changes"]
  N3["3.3 Compare containment options"]
  N4["3.4 Eradication requires a supported cause"]
  N5["3.5 Verify recovery, not merely activity"]
  N6["3.6 Guided reconciliation and return-to-service lab"]
  N7["3.7 Contained lab: validate a recovery comparison be"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX038"]

Module logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["3.1 A decision loop rather than a fixed sequence"]
  N2["3.2 The case changes"]
  N3["3.3 Compare containment options"]
  N4["3.4 Eradication requires a supported cause"]
  N5["3.5 Verify recovery, not merely activity"]
  N6["3.6 Guided reconciliation and return-to-service lab"]
  N7["3.7 Contained lab: validate a recovery comparison be"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX038"]

[Learning outcomes]
    |
    v
[3.1 A decision loop rather than a fixed sequence]
    |
    v
[3.2 The case changes]
    |
    v
[3.3 Compare containment options]
    |
    v
[3.4 Eradication requires a supported cause]
    |
    v
[3.5 Verify recovery, not merely activity]
    |
    v
[3.6 Guided reconciliation and return-to-service lab]
    |
    v
[3.7 Contained lab: validate a recovery comparison be]
    |
    v
[Worked example] --> [Exercise EX038]

Module learning outcomes

Across these five daily lessons, a learner should be able to:

1. Distinguish a temporary containment action from cause removal and verified service recovery.
2. Compare two response options against harm reduction, operational cost, reversibility, authority and evidence loss.
3. Record an approval, fallback condition and next review time for a proposed intervention.
4. Explain why a completed backup job is insufficient to claim recovery readiness.
5. Define a small set of tests that would justify returning a fictional service to normal use.
6. Screen candidate copies by provenance, integrity, freshness and data correctness without confusing an isolated import time with service recovery time.
7. Test a proposed containment control against the write paths in scope, distinguishing an observed block from a quiet log or an untested path.
8. Compare competing cause hypotheses and specify an authorized test, cause-specific correction and verification limit before claiming eradication.

Module 3 exercise rules

Use the synthetic case only. R1–R24 are available for all twelve exercises; P1/C1 and V1 are available only where an exercise explicitly calls for the Module 3 recovery lab. The Module 4 branches are not evidence at the 09:37 decision point. R17's 09:22 task list names B17/B18 only; the later B20 edit in R18 is evaluated against that already supplied list, not known to the training owner in advance. For each response, first name its record IDs and decision time; then separate observed fact from inference or proposal; finally name an authorized owner, an observable check and a limit. Submit the requested matrix, note or test plan so a reader can reproduce the reasoning from this book alone. The worked results at the end provide examples, while alternative choices pass when their assumptions and verification are explicit.

Use a compact 0–2 check per exercise: 0 if the requested artifact is missing or asserts an unsupported fact, 1 if the artifact answers the question but omits evidence, authority or an important limit, and 2 if it is evidence-linked, bounded and testable. This is a practice rubric for the draft, not a formal grade. Where a question names several fields, all are required for a 2. A claim of executed action before the branch card or of complete recovery after a failed V1 audit check must be revised even if other fields are present.

Lesson 11 — Containment options and their service cost

Week 3, day 1. Work through the teaching and the worked checkpoint before attempting the assigned exercise(s). Compare your artifact with the final worked result only afterward.

3.1 A decision loop rather than a fixed sequence

Module 2 left B19 unexplained and identified a time-sensitive preservation risk: R15 supplies a 24-hour retention duration for detailed booking-audit records, but not the policy anchor or any verified deletion deadline. As evidence changes, the incident lead needs a repeated loop: observe → assess possible harm → compare authorized options → act → verify effect → reassess. Containment aims to limit further harm while the cause remains under investigation. Eradication addresses a confirmed cause or persistence mechanism when the team can identify it. Recovery restores useful service under explicit checks and monitoring. These activities can overlap, and an intervention may have to be rolled back if its cost or side effects exceed its benefit.

NIST SP 800-61 Rev. 3 treats response and recovery as part of wider cybersecurity risk management. NIST SP 800-184 addresses recovery from cybersecurity events. The classroom decision table below is a teaching model built for Harbor Library, not a universal playbook or an assertion that NIST mandates these exact steps.

Three independent checks for a response control

A response measure may stop a suspicious operation and still create unacceptable harm. Conversely, it may keep the service available while failing to block the concerning activity. The incident lead must not call either result unqualified success.

For the Harbor Library booking service, separate three questions before selecting a control:

Protection: What exact unwanted operation should the intervention prevent? A session revocation targets use of the named session; a pause targets staff writes along defined routes. Neither automatically prevents activity through all accounts, APIs, jobs or software defects.

Legitimate service: What approved function may the intervention interrupt? Revocation may require staff to authenticate again; a write pause may move booking requests into a manual queue. The service owner must describe the tolerable disruption and how queued work will later be reconciled.

Verification: What specific evidence would show that the first two expectations were met? A change ticket marked "executed" is not an effectiveness test. A quiet audit result could mean no request was attempted, the relevant source was unavailable, the query was too narrow, or the control actually prevented a change. A useful test needs a specified target, observable attempt, appropriate source coverage and independently checked business state.

NIST SP 800-61 Revision 3 places response activities within wider cybersecurity risk-management decisions. The three-question contract here is an original teaching method, not a mandatory NIST form. It matters because containment can reduce one risk while increasing another; the record must show both.

3.2 The case changes

These additional records are invented. Use them with R1–R16; the timestamps are UTC and the records are the only authorized evidence for this exercise.

R17 and R18 make the authorized-training explanation for all edits less plausible and raise concern about ongoing harm. They still do not prove who controlled the session or why the edits occurred. R22 is limited to its range; R24 is a statement about the current evidence, not a finding that no exposure occurred. The lead should update confidence and urgency without converting uncertainty into an accusation.

3.3 Compare containment options

At 09:37 UTC, the team considers the following hypothetical options. No action has been taken in this text.

The best option depends on current activity, confidence, reversibility and the service owner's tolerance. A reasoned classroom proposal is to ask the incident lead and identity owner to authorize session revocation, preserve the available narrow records and have the service owner ready to pause staff writes if edits continue. Another team might choose the short write pause first if the identity session cannot be linked reliably to the edits. The learner must explain the decision, authority and fallback; no fixed option is automatically correct.

Recommendation, approval, execution and verification receipts

A decision is auditable only when each state has its own evidence. The distinction is practical, not linguistic:

At 09:37, a proposed session revocation could name desk-a's identified current session as its target. The incident lead and identity owner are the proposed approving roles; an identity administrator would operate the change under the fictional rules. The expected protective result is denial of subsequent use of that session. The expected legitimate-service consequence is staff reauthentication. The evidence plan must include both, plus a new bounded booking audit after execution. R19 describes the capability to revoke, not a completed revocation.

For the fifteen-minute pause, R21 provides a business tolerance statement, not the necessary approval of the actual target and window. An authorized operator would need the service owner's explicit decision and a means of verifying which staff-write paths are blocked. The manual queue also has an owner and reconciliation obligation. R21 must not be transformed into execution, even when the team favors the pause.

A verified control result has scope. If one approved test shows a selected session can no longer authenticate, it does not establish that every previous token was invalidated. If one staff route rejects a write, it does not establish that a legacy route is blocked. The existing W1 fixture examines those route-specific failures independently; this expansion adds the state and acceptance logic that should surround any such test.

Make the comparison observable

A response option is an intervention, not a verdict about the cause. Compare it with the same questions you would ask of a testable claim: What should change? What might break? How will we notice both effects? Who is allowed to make and reverse the change? A short decision table is more useful than assigning unexplained numerical risk scores:

The no-intervention option also needs a test and a deadline. “Observe” is a decision to accept possible further booking harm for a limited period in exchange for more information. It is not a neutral default. If the team cannot detect a new edit quickly, an observation window may be less defensible than the table suggests. If the audit is incomplete, “no new edit shown” must be reported with that limitation.

Preserve the narrow records that are likely to matter before a disruptive change when doing so does not dangerously prolong harm. The incident lead should record which sources were captured, which could not be captured in time and why. Waiting indefinitely for a perfect evidence set could expose more patrons; acting without an evidence note can make later review harder. A proportionate decision records both costs. The comparison follows the risk-management orientation of NIST SP 800-61 Rev. 3; the Harbor Library choices and matrix are original teaching examples.

Contained W1 lab: test whether a proposed write pause actually holds

A change ticket saying pause enabled records an operator's intention, not the control's effect. A quiet booking audit could mean the pause worked, no one tried to edit, the query omitted a path, or logging failed. To evaluate the proposed pause in section 3.3, define its claim before testing: during a bounded interval, staff-originated booking writes through both the ordinary interface and the legacy application route should be blocked, while read access and the approved manual queue remain usable. This is a service-specific classroom requirement, not a universal containment recommendation. A real team would identify all relevant routes with the service owner and test only under its change, privacy and safety rules. NIST SP 800-61 Rev. 3 frames effective response and recovery as part of risk management; the W1 test design and pass criteria are original teaching material, not a NIST procedure.

W1 is a separate fictional branch. Its special card W1-A says that, for an isolated training copy only, the fictional service owner and incident lead approve a 09:40–09:55 UTC staff-write pause test with a manual queue. This approval does not exist in R1–R24. In particular, R21 says a pause is tolerable; it does not authorize one. W1's later timestamps and observations cannot be cited as facts available to the 09:37 canonical decision or the Module 4 tabletop. No learner is to submit a write to a live service, use a real account or infer a real incident outcome from these invented rows.

Before reading the outcomes, put four questions on a test sheet: (1) Did the ordinary interface reject a labeled write and leave the isolated booking value unchanged? (2) Did the legacy route do the same? (3) Was a manual request recorded for later reconciliation rather than applied to the booking state? (4) Did a read-only request still work? Use independent before/after state beside the response and audit label. A response string alone can disagree with the resulting state; an absent audit line is not evidence of a block. The fixture supplies one trial per route, so even a clean result would support only those paths, values and times.

Card W1-B — isolated probe results. The labels W-UI, W-API, W-QUEUE and W-READ are new synthetic records. Room 2 and Room 4 are training values unrelated to B17–B20. The queue numbers are pending-item counts, not booking-room values.

Treat each row as a separate probe with its own reset training state. W-API's Room 4 does not become W-READ's starting value; the table is not a continuous production trace. The fixture intentionally makes the ordinary interface block a write while the legacy route changes a value. Therefore the broad claim all staff writes are paused fails under the stated W1 scope. It would be incorrect to average the two write outcomes into “50% effective”: the sample is neither random nor representative, and one demonstrated route failure is enough to reject the universal claim for this fixture. The manual queue's one pending item shows it recorded a request in this test, not that capacity, reconciliation or later service acceptance was verified.

Reproduce a narrow classification. The following standard-library Python program uses only in-memory invented rows. It does not connect to an application or change a file. It validates unique probe IDs and recognized kinds, then gives state change precedence over the service's claimed denial for write probes. The four status labels are observations about this fixture, not a production certification.

First enter the W1 fixture:

PROBES = [
    ("W-UI", "write", "rejected", "Room 2", "Room 2", "deny"),
    ("W-API", "write", "accepted", "Room 2", "Room 4", "allow"),
    ("W-QUEUE", "queue", "queued", "0 pending", "1 pending", "queue"),
    ("W-READ", "read", "returned Room 2", "Room 2", "Room 2", "read"),
]

Check the measurement, not just the classifier. A Room 2 → Room 2 pair is useful only if both reads refer to the same training booking and an authoritative state view, with a recorded read time and a known consistency rule. A cached page could remain unchanged even after a successful write. Likewise, an audit label needs a request or event pointer and a collection cutoff; a delayed deny row could describe a different attempt. W1 supplies simplified independent values for the classroom method, but does not provide those production-grade provenance fields. This is why the code names one outcome BLOCKED_IN_THIS_TEST, rather than blocked by the pause.

A pre-pause baseline is another missing control. If the W-UI test identity lacked write permission even before 09:40, a rejection during the pause would not show that the pause caused it. An authorized isolated rehearsal could first demonstrate that the same synthetic identity and route can perform a permitted write, reset the training state, then apply the approved pause and repeat the labeled probe. The test sheet would retain both request IDs, approval, state snapshots and timing. W1 does not supply such a baseline, so do not invent one in the worked result. The W-API state change is still enough to reject the broad pause claim within the stated W1 fixture, assuming its supplied independent state is reliable. In a real investigation, the owners would verify that assumption before acting on the comparison.

Then run the classifier in the same scratch Python session. These two labeled listings form one program; the second uses PROBES from the first.

def classify(kind, response, before, after, audit):
    if kind == "write":
        if before != after:
            return "WRITE_PATH_NOT_BLOCKED"
        if response == "rejected" and audit == "deny":
            return "BLOCKED_IN_THIS_TEST"
        return "INCONCLUSIVE_WRITE"
    if kind == "queue":
        if (response == "queued" and before == "0 pending"
                and after == "1 pending" and audit == "queue"):
            return "QUEUE_ITEM_RECORDED"
        return "INCONCLUSIVE_QUEUE"
    if kind == "read":
        if (response == "returned Room 2" and before == after
                and audit == "read"):
            return "READ_RETURNED_EXPECTED_VALUE"
        return "INCONCLUSIVE_READ"
    raise ValueError("unknown probe kind")


seen = set()
results = {}
for probe_id, kind, response, before, after, audit in PROBES:
    if probe_id in seen:
        raise ValueError("duplicate probe ID")
    seen.add(probe_id)
    results[probe_id] = classify(kind, response, before, after, audit)
    print(f"{probe_id}: {results[probe_id]}")

write_ids = [probe_id for probe_id, kind, *_ in PROBES if kind == "write"]
claim = "SUPPORTED_FOR_TESTED_PATHS" if all(
    results[probe_id] == "BLOCKED_IN_THIS_TEST" for probe_id in write_ids
) else "NOT_ESTABLISHED"
print(f"write_pause_claim: {claim}")

The final claim line deliberately says SUPPORTED_FOR_TESTED_PATHS even in a hypothetical all-blocked variant. It would still say nothing about undiscovered routes, future time windows, queue capacity or why the earlier anomalous edits occurred. In the printed W1 fixture the output is NOT_ESTABLISHED; the legacy-route state change is a concrete failure of the proposed write-pause claim in the isolated test. The incident lead should keep the result as a failed validation, ask the service owner to correct or narrow the control design, and require a new authorized test before relying on it. A manual queue may reduce service disruption, but it does not compensate for an uncontrolled write route.

W1-SCOPE supplemental check — required route and service coverage

The original W1 classifier gives a correct result for its complete fixture, but its final all() test silently accepts missing required routes: with only W-UI or with no rows, it can print SUPPORTED_FOR_TESTED_PATHS. A read response can also name Room 2 when the independent value is actually Room 4. These are defects in the teaching evaluator, not observations of an actual control. Run the following supplemental audit after both original W1 listings in the same scratch session. It does not alter the five original output lines. Here W-UI and W-API are the two required write routes; W-QUEUE and W-READ are separate legitimate-service functions.

# W1-SCOPE supplemental audit; requires the original PROBES and classify().
EXPECTED = {
    "W-UI": "write", "W-API": "write",
    "W-QUEUE": "queue", "W-READ": "read",
}

def w1_gates(probes):
    ids = [row[0] for row in probes]
    if len(ids) != len(set(ids)):
        return ("PROTECTION_UNEVALUABLE_DUPLICATE_ID",
                "SERVICE_UNEVALUABLE_DUPLICATE_ID")
    rows = {row[0]: row for row in probes}
    if any(k not in EXPECTED or row[1] != EXPECTED.get(k)
           for k, row in rows.items()):
        return ("PROTECTION_UNEVALUABLE_UNEXPECTED_PROBE",
                "SERVICE_UNEVALUABLE_UNEXPECTED_PROBE")

    if not {"W-UI", "W-API"}.issubset(rows):
        protection = "PROTECTION_UNEVALUABLE_MISSING_ROUTE"
    elif all(classify(*rows[k][1:]) == "BLOCKED_IN_THIS_TEST"
             for k in ("W-UI", "W-API")):
        protection = "PROTECTION_TESTED_PATHS_PASS_NOT_CAUSAL"
    else:
        protection = "PROTECTION_TESTED_PATHS_FAIL"

    if not {"W-QUEUE", "W-READ"}.issubset(rows):
        service = "SERVICE_UNEVALUABLE_MISSING_FUNCTION"
    else:
        q, r = rows["W-QUEUE"], rows["W-READ"]
        queue_ok = classify(*q[1:]) == "QUEUE_ITEM_RECORDED"
        read_ok = (
            classify(*r[1:]) == "READ_RETURNED_EXPECTED_VALUE"
            and r[2] == "returned Room 2"
            and r[3] == "Room 2"
            and r[4] == "Room 2"
        )
        service = ("SERVICE_TESTED_FUNCTIONS_PASS"
                   if queue_ok and read_ok
                   else "SERVICE_TESTED_FUNCTIONS_FAIL")
    return protection, service

def print_w1_gate(label, probes):
    print(f"{label}: {' | '.join(w1_gates(probes))}")

both = [
    ("W-API", "write", "rejected", "Room 2", "Room 2", "deny")
    if row[0] == "W-API" else row
    for row in PROBES
]
wrong_read = [
    ("W-READ", "read", "returned Room 2", "Room 4", "Room 4", "read")
    if row[0] == "W-READ" else row
    for row in both
]

print_w1_gate("ORIGINAL_W1", PROBES)
print_w1_gate("LEGACY_ROW_MISSING",
              [row for row in PROBES if row[0] != "W-API"])
print_w1_gate("NO_ROWS", [])
print_w1_gate("BOTH_WRITES_BLOCKED", both)
print_w1_gate("WRONG_READ_STATE", wrong_read)
print_w1_gate("QUEUE_ROW_MISSING",
              [row for row in both if row[0] != "W-QUEUE"])

Predict the six additional results before looking at the W1-SCOPE worked result. An absent route is unevaluated, never blocked. Even when both named writes are blocked in this synthetic fixture, the result is not causal evidence of the proposed pause without a permitted pre-pause baseline, request-linked audit evidence and authoritative before/after reads. The queue test checks admission of one item, not eventual booking fulfillment, reconciliation capacity or recovery acceptance. No code here performs a live write or service call.

Practice submission. Prepare (1) a four-row table of status and source fields; (2) a one-sentence containment claim with its scope and test window; (3) a decision record saying whether to rely on the pause, who must approve a change and how the manual item will be reconciled; and (4) a counterfactual W1-V in which W-UI's response remains rejected but its after-state becomes Room 4. Predict the new W-UI classification and the overall claim without changing any other field. The model output and a bounded decision appear in the final worked-results appendix. Seven-point rubric: one point for each of the four probe classifications, one for rejecting the broad pause claim with the W-API state evidence, one for a bounded owner-approved corrective and reconciliation plan, and one for the W1-V prediction. A 7/7 response demonstrates correct reasoning on W1, not verified containment of a real service.

RB3 — Preparing and verifying containment reversal

RB3.1 Containment introduces a second risk that must be managed

Containment is intended to limit a particular harmful path. It can simultaneously interrupt ordinary work, prevent approved users from accessing a necessary function, or fail to block the activity the incident team intended to stop. A team that can enable a control but has no prepared way to reconsider or reverse it may turn a temporary protective measure into a prolonged service problem.

The existing §3.3 compares current-session revocation, a short staff-write pause and observation. It already identifies business costs and possible fallbacks. This expansion addresses the next, distinct question: if a selected intervention produces unacceptable side effects or fails its expected protection, what evidence and authority are needed to reverse that intervention without silently reintroducing the original risk?

A reversal can involve restoring an earlier configuration, stopping a temporary restriction, replacing a control with a narrower one, or moving to a separately approved operational mode. The word rollback is useful, but not every action can be literally undone. A revoked identity session may need a new, appropriately verified authentication procedure; restoring an old configuration may reintroduce the same vulnerability that containment was intended to address. Undoing an emergency pause may permit writes again without proving that the original suspicious edit path is now safe.

NIST SP 800-61 Rev. 3 connects incident response and recovery with risk management throughout an incident. NIST SP 800-184 addresses planned recovery, testing and improvement. These sources support careful planning and evaluation; the precise RB3 reversal contract is an original OpenCyber Academy teaching construct, not a mandatory NIST workflow.

RB3.2 Write down the purpose of reversal before selecting a command

The phrase "undo the containment" conceals at least two different reasons for taking action.

Service-harm trigger. The response team has a reliable, scoped observation that the control disrupted legitimate work beyond the accepted operating limit. For example, a fictional short staff-write pause may unexpectedly interfere with an essential permitted function. The proposed next step is not automatically "remove all protection"; it may be to restore that function under a narrower, separately approved safety condition.

Protection-failure trigger. A credible post-action test finds that the targeted harmful activity remains possible in the declared control scope. A responder might reasonably request replacement, correction or withdrawal of an ineffective control. However, reverting without an alternative could make the situation worse. The team needs to compare the present control, proposed reversal and remaining safeguards.

Both triggers need an observable condition and a source that could be challenged. "The desk is unhappy" is not the same as a verified failure of the service owner's accepted requirements. "The logs are quiet" is not proof that protection worked if no meaningful attempt or source coverage was established. A negative result can be informative only within the test's inputs and scope.

The intervention decision and the reversal decision are separate. Even if a change was previously approved, its inverse need not be automatically approved. The risk has changed, the available evidence may have changed and the accountable owner must understand the new tradeoff.

At rapid level, the learner should describe the trigger in operational language: which function or expected protective result failed, how was that observed, and what is the narrowest next decision it justifies?

RB3.3 Establish a verified pre-intervention state, not an idealized past

A useful reversal plan starts before applying the original control. It records the last verified configuration or state relevant to the proposed change: the affected component, configuration version, tested legitimate service behavior, observed protective posture, dependencies, owner and source of the baseline record.

The phrase "restore the old settings" is unsafe without a trustworthy target. The old version may be missing, incomplete, incompatible with a newer dependency or already known to permit the harmful behavior. A baseline should be an identifiable, owner-reviewed candidate for return, not a promise of correctness.

The baseline card should answer:

For a service that has been changing during an incident, simply replaying a saved configuration may be inappropriate. A baseline can be known at capture time yet not safe to reapply later. The reviewer must check its current applicability and the intervening system changes.

The RB3 lab uses the entirely invented identifier CFG-GOOD-7 and a Boolean baseline_verified. Neither is evidence that a real baseline exists. The positive fixture models the discipline of requiring a known starting reference; it does not check the contents or authenticity of an actual configuration.

RB3.4 Define the original intervention as an exact bounded change

A reversal can only be compared with an intervention if the first change is itself traceable. The original action record needs its own proposal, approval, object scope, effective window, operator receipt, expected protection and service impact. Otherwise the team may not know what needs undoing.

Suppose a containment proposal affects only a synthetic staff-write service function. A later reversal may legitimately aim to restore only that function. A request to alter identity settings at the same time would be an expanded action, not an automatic inverse of the original. That expansion requires separate justification and authorization.

The RB3 fixture represents the original control as CTRL-PAUSE-9 with affected scope staff-write. Its fictional prior approval is a declared input. The baseline restoration target is CFG-GOOD-7. These labels are not Harbor Library configuration IDs.

An implementation plan should preserve both what the operator intended to change and what was actually reported changed. A control may be applied partially or to the wrong object. In that situation, executing a generic inverse could affect components the original operation never reached.

The safe preparation step is to identify a precise rollback target and its permitted scope, plus a stop condition if the current state no longer matches the assumptions in the plan. A preapproved recipe for the wrong current environment is not automatically safe.

This is deliberately different from W1, which checks specific write routes under a pause. RB3 does not inject writes, run a gateway test or determine whether W1's legacy route was blocked. It checks whether the proposed inverse has a reviewable evidence chain.

RB3.5 Approval must name the reversal, its window and its business cost

A rollback request should identify the specific control being reconsidered, the intended target state, the components or functions affected, the justification, the risks reintroduced and the authorization window. An approval for the original pause does not necessarily approve ending that pause, and R21's tolerance statement cannot be used as approval for any live change.

The accountable owner for a service-affecting reversal may be the service owner under the applicable fictional governance rule, coordinated with the incident lead. Identity owners may be needed if the intervention concerns authentication rather than a staff-write setting. The operator applies the change only after the relevant decision; the reviewer checks the result independently.

For a clear approval record, name the operation, not merely the ticket: "revert the specified synthetic staff-write control to the reviewed candidate state within the defined window, subject to the stated protective and business checks." The record should include:

• The approved target and exact component/function set.
• The allowed start/end of execution, using an agreed clock.
• The decision owner and the operator.
• The triggering evidence and reason for reversal.
• The business impact expected if the inverse is performed.
• The compensating safeguard that should remain effective.
• The post-reversal verification, reviewer and failure response.
• The planned human reassessment, even if the initial check passes.

A missing or expired approval is a hold condition, not a license to treat urgency as permission. If immediate harm creates a real emergency, an organization might have a separately governed emergency change procedure; the training exercise does not invent such powers.

RB3 checks a small fictional approved window, 14:05–14:25 UTC on an entirely invented date. The clock is a consistency input, not a measured Harbor Library outage or recovery objective.

RB3.6 Specify a trigger that can be falsified

The team should be able to say what observation would cause a planned reversal to be requested and what observation would show that the trigger did not occur.

For a service-harm trigger, an owner-approved isolated measurement might show that a previously permitted legitimate task is now unavailable because of the controlled change, beyond accepted limits. The analyst should compare the affected service function with its baseline and account for other possible causes. If the function was already unavailable before the intervention, a post-action failure cannot by itself establish that the control caused the problem.

For a protection-failure trigger, an approved, bounded post-action check may show that the threat path remains available under the tested scope. The team should not interpret a quiet or incomplete log as a successful block. Equally, an unrelated failure does not prove the targeted protection failed.

A valid trigger record should name:

1. The condition being tested.
2. The source and input of the observation.
3. The time and operating scope.
4. The expected result and observed difference.
5. The custodian or owner confirming the relevant semantics.
6. A limit: what the observation cannot establish about other routes or business outcomes.

The RB3 fixture uses either SERVICE_HARM or PROTECTION_FAILURE as the artificial trigger kind, with a corresponding training receipt. These are declared exercise observations, not detector outputs or events added to R1–R24.

The negative variant TRIGGER_NOT_OBSERVED deliberately removes the supporting observation. The validator then refuses to convert a plausible rollback proposal into an executed action.

RB3.7 Proposed reversal, approved reversal, executed reversal and checked effect

The same action-state discipline used elsewhere in Module 3 applies to the reversal as a new operation. A team should not merge the original control ticket with the inverse action and then declare both successful.

The order matters. A ticket with a return configuration attached is still a plan. An approval receipt is still not an execution receipt. A command or workflow reporting success may have operated on the wrong object, changed only one dependency or left the actual service in an unexpected state. Record the operator's actual execution time separately from the approval time; an action reported outside the approved window, or before the decision, is not covered by that approval.

RB3 adds a specific target-matching check. Its execution receipt must refer to the declared candidate configuration rather than an unrelated version. A variant with an execution receipt for CFG-UNKNOWN must fail even if the other synthetic status flags are positive.

After execution, the team needs an observation of the resulting state and two different acceptance checks. Those are discussed next. The presence of a receipt does not replace the effect observation.

This section does not repeat G3-DG's generic decision-gate tutorial. It applies those distinctions to the risk-bearing inverse operation, including the verified baseline, trigger and competing post-reversal outcomes.

RB3.8 Verify protective behavior after rollback, not just restored availability

Removing a disruptive restriction may restore legitimate service while also reintroducing the harmful path that containment was intended to limit. A successful rollback command is therefore not sufficient, and even a successful booking operation may be too narrow.

The post-reversal protection check asks: Under the new, declared operating mode, what prevents or detects the selected unacceptable action, and what bounded observation supports that claim? The original control may have been removed, so the reviewer cannot simply require the exact same control to remain enabled. Instead, the service owner and security lead must define an acceptable residual protection condition, perhaps through a narrower approved safeguard or restricted mode.

A failed protection check means the reversed system cannot be called acceptably protected for the tested purpose. It does not always mean the inverse had no technical effect. It can mean that legitimate service resumed but the protective risk is unacceptable.

The RB3 negative case PROTECTION_REGRESSED models that outcome. It returns PROTECTION_NOT_ACCEPTED even when an execution receipt and a positive legitimate-service check are present. This prevents the seductive conclusion "rollback succeeded because the desk can work again."

Do not interpret this as a universal instruction to keep every harmful control in place indefinitely. The accountable owners must balance risks and may select a different approved containment strategy or restricted service mode. The lesson is to document and verify that tradeoff, not to choose the outcome for them.

No real security-control test is performed in RB3. The protection_check field is an invented Boolean and cannot establish that a live system is safe.

RB3.9 Verify legitimate service after rollback, not merely configuration identity

The inverse operation may report that a previous configuration was restored while the original business problem remains. A dependency may still be unavailable; a required approval workflow may still fail; data may still be wrong; or the configuration switch may not control the user-facing function the team intended to restore.

The post-reversal legitimate-service check asks whether the specific owner-approved permitted function works in the declared scope. It should use a relevant positive task and a correct state result, not a generic process-health indicator.

The exact result depends on the intervention. For a fictional staff-write pause, the owner may care whether approved work can resume through the permitted route after the separate queue and booking-state conditions are satisfied. For a session revocation, the owner may care whether legitimate staff can authenticate through the approved replacement procedure. Those are different operations; neither is safely "undone" by reversing a single string.

The RB3 variant LEGITIMATE_SERVICE_DOWN declares that the post-reversal permitted function still fails. It returns SERVICE_NOT_ACCEPTED even though the simulated protective check passes. This demonstrates that protection and business availability are independent acceptance conditions.

A configuration equality test also cannot prove full booking-state correctness. An application can run under an old configuration while a booking database still contains unresolved inconsistencies. Later P1/C1 and V1 belong to their own separately released cards, and RB3 never changes their findings.

The correct report is "post-reversal legitimate function not accepted in this fixture," not "the entire library remains down" or "the data were destroyed."

RB3.10 A valid reversal can still leave open human decisions

Why not mark everything recovered when both post-reversal checks pass? Because the checks cover only their stated function and scenario. The original incident may have an unresolved cause, untested paths, preservation needs, pending communication decisions and business-state errors that the inverse operation did not address.

A safe reversal record keeps a human review item explicitly open with an owner and a next checkpoint. This is not a cosmetic ticket. It can include verifying residual exposure, choosing whether another control is needed, reconciling operations deferred during containment, and deciding whether a restricted mode remains acceptable.

The RB3 positive case returns BOUNDED_REVERSAL_CHECKED_REVIEW_OPEN. The final words are intentional: the simulated reversal met two bounded post-state checks, but it is not incident closure. A negative case with human_review_open=False fails because the teaching contract requires an accountable human reassessment even after technically favorable observations.

The output must not be equated with RTO or RPO attainment. Module 3's RTE3 exercise measures scoped interruption-to-acceptance under a different contract; RB3 has no completed service-owner recovery-acceptance timeline. This is a reversal readiness and verification model.

A team may legitimately leave a temporary restriction partially active while awaiting review; if so, it must say which service and protective conditions are still in effect. Silent assumption of normal service is not an acceptable outcome.

RB3.11 Full wrong example: automatically undoing a harmful pause

Synthetic classroom contrast; none of these events occurred at Harbor Library's 09:37 checkpoint.

Imagine a fictional application operator learns that a temporary staff-write control is slowing legitimate booking work. A ticket says the earlier control was "deployed," and someone remembers a configuration called CFG-GOOD-7. The operator immediately restores that version and announces: "Rollback succeeded, the service is back, and we can close the incident."

This is wrong, even if restoring the old configuration appears to make one staff screen work. The team has not established that the old version is the verified relevant baseline, that reversal was authorized for the precise current state, or that a defined trigger was confirmed. It also has not recorded an execution receipt identifying the exact target, checked whether the protective goal still holds, or obtained the service owner's acceptance of the legitimate user outcome.

The more subtle failure is that reversing the control may expose the same path the control had been introduced to restrict. A restored booking interface could be responsive and still permit unintended edits. Conversely, the old configuration may fail to restore the intended function because of another dependency.

A superficially reassuring case note would compound the error by replacing the original proposal and intervention history with one line reading "fully recovered." That prevents a later reviewer from understanding what happened and what the reversible risk actually was.

Correct handling of this wrong case: Hold any further unapproved change, preserve the relevant source and configuration history, identify the precise previously verified state, seek the accountable rollback decision, specify both protection and legitimate-service checks, and record open human review. If the first change has already occurred under an unverified route, the team must document that fact and obtain an owner-governed safe next decision. It must not invent a retroactive approval.

RB3.12 Full correct example: a reviewed, narrow reversal with residual risk open

Independent RB3 invented variant, dated 2040-04-10; not canonical Harbor Library evidence.

A fictitious service owner had an isolated, previously verified candidate configuration CFG-GOOD-7, documented for the narrow function staff-write. A separately approved classroom control CTRL-PAUSE-9 later restricted that function. A named training observation TRIG-SVC-01 reports that the required permitted service activity is now failing under the isolated control and meets the declared SERVICE_HARM trigger.

The fictional service owner, ServiceOwnerSim, considers a scoped reversal request. The request names only the original staff-write function, specifies the approved candidate CFG-GOOD-7, explains the loss-of-protection risk, and defines an authorization window of 14:05–14:25 UTC. At 14:12, the invented approval receipt APP-RB-01 is present in the fixture. At 14:13, the operator's simulated execution receipt is EXEC-RB-01 for the expected target, within the approved window; a separate declared post-state observation names that target. The earlier control also has a distinct application receipt. None authenticates an actual change.

The two different post-reversal checks are then considered: (1) the selected protective condition remains acceptable in the tested scope under a defined residual safeguard, and (2) the legitimate staff function meets its declared business check. Only if both are supported in the fixture does the validator return BOUNDED_REVERSAL_CHECKED_REVIEW_OPEN.

That verdict is deliberately narrow. It does not authenticate a real approver, prove the earlier intervention's cause, test every write route, reconcile every pending booking or close the incident. The service owner must still review residual risks, choose any further restriction or monitoring and preserve an auditable next checkpoint.

A second positive scenario changes the trigger to PROTECTION_FAILURE, with an independent artificial trigger receipt. It can reach the same bounded result if the new operating mode still meets both acceptance checks. This illustrates that the reason for reconsideration can differ while the authorization and verification discipline remains necessary.

A full report preserves the original baseline reference, intervention approval, trigger, new rollback approval, receipt, two check results and open human-review status as separate historical entries.

RB3.13 Negative evidence and the need to stop before compounding harm

A failed condition can call for holding a reversal proposal, reassessing an already reported action, or escalating immediately to the appropriate owner depending on when it is discovered.

If approval is absent, an operator must not treat the proposal as executable merely because service disruption is costly. If the scope exceeds the original control, the extra object needs its own authorization and safe test; it must not be folded into the inverse. If the trigger was not observed, the evidence for change is unestablished. The team should recheck the source before deciding that rollback is justified.

If a proposal lacks an execution receipt, report proposed/not performed. If an execution receipt exists without post-state observation, report execution claimed but effect unverified. A command result is not a service test.

If the protection check fails after the reversal, the team may need a revised, owner-approved containment strategy. If the legitimate-service check fails, the inverse did not accomplish its business objective in the tested scope, even if protection remains intact. Neither failure licenses uncontrolled trial-and-error changes against production.

An unknown baseline or expired window is also significant. An inverse plan that was valid against one configuration may no longer be safe when state or dependencies have changed. An expired approval does not silently renew itself because the operator is already familiar with the system.

This is why rollback readiness should be prepared alongside containment, not improvised after a failed intervention. A team needs an authorized response even when its first solution makes conditions worse.

RB3.14 A reversal card must preserve history, not overwrite the initial decision

A reviewable reversal card should link three distinct records: the pre-intervention baseline, the original intervention and the proposed or executed reversal. Each has its own evidence and authorization.

A useful field-level record includes:

In the canonical 09:37 decision, no reversal record exists because no intervention was executed. A future optional synthetic exercise may introduce one, but its identifiers must remain outside R1–R24 and the later Branch S/P story.

The correct practice is to append the reversal record as a new dated activity rather than replacing the Round C proposal or asserting that a later action had already happened. This preserves the distinction between history and hindsight.

RB3.15 Preparing the next safe action when a reversal fails

What should the learner propose if an RB3 negative case appears?

For a missing approval or unauthorized scope, identify the accountable owner and request a corrected, bounded decision. For an unknown baseline, ask the configuration or service custodian to establish a safe candidate and document the comparison. For a missing trigger receipt, obtain a genuine authorized observation rather than inventing the incident. For an execution receipt without independent effect evidence, request the appropriate bounded check before claiming success.

If protection fails after rollback, the next step is not automatically to reinstate the original harmful control. The owner must compare the original intervention, proposed alternative and current service risk. A narrower control, monitored restricted mode or another approved protective action may be appropriate, but the supplied RB3 fixture does not select a real solution.

If legitimate service remains down after rollback, the result should be escalated with the specific failed function and source rather than declared successful because the configuration identifier matches the baseline. Additional fault diagnosis or a separately approved recovery action may be necessary.

For all these cases, preserve current evidence, avoid adding unexplained changes and assign an owner and review trigger. An apparently successful inverse is only one step of managing an incident; unresolved cause, correct booking state and actual user acceptance remain separate tasks.

RB3.16 Teaching synthesis and relationship to other Module 3 laboratories

W1 tests whether a proposed write pause actually blocks the stated isolated routes. G3-DG distinguishes generic approval, execution and acceptance states. MQ3 checks duplicate/manual-queue processing decisions. RTE3 measures scoped time and data recovery evidence. RB3 instead tests whether a new inverse of a disruptive or ineffective intervention has a verified baseline, bounded new authority, observed trigger, exact execution target, independent post-reversal protective and business checks, and explicit human review.

These questions are complementary, not interchangeable. A successful control test is not a safe rollback plan; a successful rollback receipt is not verified protection; a normal-looking service function is not full recovery; and an open review item is not an incident-closure receipt.

A rapid-course learner has met this section's objective when they can decide whether to hold, propose, verify or escalate a reversal from a small evidence packet, explain why two independent post-state checks matter, and refuse to insert the synthetic scenario into the canonical 09:37 decision.

RB3 offline laboratory — Containment Reversal Readiness and Verification

Classification: Independent, entirely synthetic laboratory. All identifiers, times, flags, approvals, execution receipts and post-state results are invented for RB3. Nothing relates to actual Harbor Library operations, R1–R24, Branch S/P or other labs.

Objective: Evaluate a proposed inversion of a fictional isolated staff-write containment setting. Accept a narrowly valid rehearsal only when pre-state, prior intervention, new reversal approval, scope, execution window, observed trigger, matching execution receipt and both post-reversal outcomes are represented, with a human review remaining open.

Environment: Python 3 standard library, in memory. No files, network, accounts, logins, real controls, destructive commands, configuration updates or production access.

Inputs and expected rule

The fictitious date is 2040-04-10 UTC. The verified pre-state reference is CFG-GOOD-7; the prior approved and reported-applied containment reference is CTRL-PAUSE-9, affecting only staff-write. Its application receipt is a declared fixture value, not an authenticated event. Reversal requires a separate ServiceOwnerSim approval receipt and a 14:05–14:25 UTC window, and the proposed rollback targets that same narrow function. A service-harm or protection-failure observation is needed before an inverse is pursued. The operator's execution time must follow the approval decision and fall inside its window. The returned target and the separately observed post-state target must match the plan. The trigger receipt must agree with the declared trigger kind.

In this controlled fixture, protection_check=True means a declared residual protective check passed for the tested scope after the inverse. It does not mean the original blocking control remained active. legitimate_service_check=True means the declared permitted business check passed. Neither Boolean proves a real system property. A positive result must retain human_review_open=True.

Full executable Python 3 program

from copy import deepcopy
from datetime import datetime

def utc(value):
    return datetime.fromisoformat(
        "2040-04-10T" + value + ":00+00:00"
    )

BASE = {
    "baseline_ref": "CFG-GOOD-7",
    "baseline_verified": True,
    "intervention_id": "CTRL-PAUSE-9",
    "intervention_approved": True,
    "intervention_execution_receipt": "EXEC-CTRL-01",
    "intervention_scope": {"staff-write"},
    "rollback_target": "CFG-GOOD-7",
    "rollback_scope": {"staff-write"},
    "rollback_owner": "ServiceOwnerSim",
    "rollback_approver": "ServiceOwnerSim",
    "rollback_approval_receipt": "APP-RB-01",
    "authorized_window": (utc("14:05"), utc("14:25")),
    "decision_time": utc("14:12"),
    "execution_time": utc("14:13"),
    "trigger_kind": "SERVICE_HARM",
    "trigger_observed": True,
    "trigger_receipt": "TRIG-SVC-01",
    "rollback_proposed": True,
    "execution_receipt": "EXEC-RB-01",
    "executed_target": "CFG-GOOD-7",
    "post_state_observed": True,
    "observed_target": "CFG-GOOD-7",
    "protection_check": True,
    "legitimate_service_check": True,
    "human_review_open": True,
}

def evaluate(p):
    if not p["baseline_verified"] or not p["baseline_ref"]:
        return "HOLD_BASELINE_UNVERIFIED"
    if (
        not p["intervention_approved"]
        or not p["intervention_execution_receipt"]
    ):
        return "HOLD_PRIOR_INTERVENTION_UNVERIFIED"
    if (
        not p["rollback_approval_receipt"]
        or p["rollback_approver"] != p["rollback_owner"]
    ):
        return "HOLD_APPROVAL_MISSING_OR_WRONG"
    if (
        not p["rollback_scope"]
        or not p["rollback_scope"].issubset(
            p["intervention_scope"]
        )
    ):
        return "HOLD_ROLLBACK_SCOPE_INVALID"
    start, end = p["authorized_window"]
    if not (start <= p["decision_time"] < end):
        return "HOLD_OUTSIDE_APPROVED_WINDOW"
    if (
        not p["trigger_observed"]
        or not p["trigger_receipt"]
        or p["trigger_kind"] not in {
            "SERVICE_HARM", "PROTECTION_FAILURE"
        }
        or p["trigger_receipt"] != {
            "SERVICE_HARM": "TRIG-SVC-01",
            "PROTECTION_FAILURE": "TRIG-PROT-02",
        }.get(p["trigger_kind"])
    ):
        return "HOLD_TRIGGER_UNOBSERVED"
    if (
        not p["rollback_proposed"]
        or not p["execution_receipt"]
    ):
        return "PROPOSED_NOT_EXECUTED"
    if (
        p["execution_time"] is None
        or not (p["decision_time"] <= p["execution_time"] < end)
    ):
        return "HOLD_EXECUTION_TIME_INVALID"
    if (
        p["rollback_target"] != p["baseline_ref"]
        or p["executed_target"] != p["rollback_target"]
    ):
        return "EXECUTION_TARGET_MISMATCH"
    if not p["post_state_observed"]:
        return "EXECUTION_REPORTED_EFFECT_UNVERIFIED"
    if p["observed_target"] != p["rollback_target"]:
        return "POST_STATE_TARGET_MISMATCH"
    if p["protection_check"] is not True:
        return "PROTECTION_NOT_ACCEPTED"
    if p["legitimate_service_check"] is not True:
        return "SERVICE_NOT_ACCEPTED"
    if not p["human_review_open"]:
        return "HUMAN_REVIEW_MISSING"
    return "BOUNDED_REVERSAL_CHECKED_REVIEW_OPEN"

CASES = [
    ("VALID_WITH_OPEN_REVIEW", {}),
    (
        "VALID_AFTER_PROTECTION_TRIGGER",
        {
            "trigger_kind": "PROTECTION_FAILURE",
            "trigger_receipt": "TRIG-PROT-02",
        },
    ),
    ("BASELINE_UNKNOWN", {"baseline_verified": False}),
    (
        "PRIOR_INTERVENTION_UNVERIFIED",
        {"intervention_approved": False},
    ),
    ("PRIOR_INTERVENTION_NO_EXECUTION", {"intervention_execution_receipt": None}),
    ("APPROVAL_ABSENT", {"rollback_approval_receipt": None}),
    ("WRONG_APPROVER", {"rollback_approver": "AnalystSim"}),
    (
        "SCOPE_EXCEEDED",
        {"rollback_scope": {"staff-write", "identity"}},
    ),
    ("EMPTY_SCOPE", {"rollback_scope": set()}),
    ("WINDOW_EXPIRED", {"decision_time": utc("14:30")}),
    ("EXECUTION_OUTSIDE_WINDOW", {"execution_time": utc("14:30")}),
    ("EXECUTION_BEFORE_DECISION", {"execution_time": utc("14:11")}),
    ("TRIGGER_NOT_OBSERVED", {"trigger_observed": False}),
    ("TRIGGER_NOT_SUPPORTED", {"trigger_kind": "RUMOR"}),
    ("TRIGGER_RECEIPT_MISMATCH", {"trigger_kind": "PROTECTION_FAILURE"}),
    ("PROPOSAL_WITHOUT_EXECUTION", {"execution_receipt": None}),
    ("RECEIPT_WRONG_TARGET", {"executed_target": "CFG-UNKNOWN"}),
    ("RECEIPT_WITHOUT_EFFECT", {"post_state_observed": False}),
    ("POST_STATE_WRONG_TARGET", {"observed_target": "CFG-UNKNOWN"}),
    ("PROTECTION_REGRESSED", {"protection_check": False}),
    (
        "LEGITIMATE_SERVICE_DOWN",
        {"legitimate_service_check": False},
    ),
    ("HUMAN_REVIEW_NOT_OPEN", {"human_review_open": False}),
]

for name, changes in CASES:
    candidate = deepcopy(BASE)
    candidate.update(changes)
    print(f"{name}: {evaluate(candidate)}")

assert (
    evaluate(BASE)
    == "BOUNDED_REVERSAL_CHECKED_REVIEW_OPEN"
)

RB3-RECHECK supplemental audit — bind receipts to time and scope

The baseline RB3 validator checks declared values but cannot inspect whether an approval was actually recorded before execution, whether a trigger was observed before approval, or whether the two post-state checks occurred after execution. A positive baseline label therefore does not establish that sequence. In a fresh RB3-only scratch session, run this supplement after the full 22-case program. Its receipts are new, invented classroom inputs; they are not R-series facts, authenticated records, or W1 observations. approval_time == decision_time and execution_receipt_time == execution_time are explicit assumptions of this small fixture, not universal operational rules.

# RB3-RECHECK: requires original BASE, evaluate(), utc().
from copy import deepcopy

POSITIVE = "BOUNDED_REVERSAL_CHECKED_REVIEW_OPEN"
RECEIPTS = {
    "approval_ref": "APP-RB-01",
    "approval_owner": "ServiceOwnerSim",
    "approval_scope": {"staff-write"},
    "approval_target": "CFG-GOOD-7",
    "approval_time": utc("14:12"),
    "trigger_time": utc("14:10"),
    "execution_receipt_time": utc("14:13"),
    "post_state_time": utc("14:14"),
    "protection_checked_time": utc("14:15"),
    "service_checked_time": utc("14:16"),
    "review_as_of": utc("14:17"),
}

def rb3_recheck(packet, receipt):
    original = evaluate(packet)
    if original != POSITIVE:
        return original
    required = set(RECEIPTS)
    if (set(receipt) != required or
            any(receipt[key] is None for key in required)):
        return "RECEIPT_METADATA_INCOMPLETE"

    if (
        receipt["approval_ref"] != packet["rollback_approval_receipt"]
        or receipt["approval_owner"] != packet["rollback_owner"]
        or receipt["approval_scope"] != packet["rollback_scope"]
        or receipt["approval_target"] != packet["rollback_target"]
    ):
        return "APPROVAL_NOT_BOUND_TO_TARGET_AND_SCOPE"

    start, end = packet["authorized_window"]
    if not (
        receipt["trigger_time"] <=
        receipt["approval_time"] == packet["decision_time"]
        and start <= receipt["approval_time"] <=
        receipt["execution_receipt_time"] == packet["execution_time"] < end
    ):
        return "TRIGGER_APPROVAL_EXECUTION_ORDER_INVALID"

    if not (
        packet["execution_time"] <=
        receipt["post_state_time"] <=
        receipt["protection_checked_time"] <= receipt["review_as_of"]
        and receipt["post_state_time"] <=
        receipt["service_checked_time"] <= receipt["review_as_of"]
    ):
        return "POST_EXECUTION_EVIDENCE_TIME_INVALID"

    return "BOUNDED_RECHECK_WITH_DATED_RECEIPTS_REVIEW_OPEN"

def print_rb3_recheck(label, packet, changes):
    receipt = deepcopy(RECEIPTS)
    receipt.update(changes)
    print(f"{label}: {rb3_recheck(packet, receipt)}")

print_rb3_recheck("DATED_BASELINE", BASE, {})
print_rb3_recheck("UNRECORDED_CHECKS", BASE,
                  {"protection_checked_time": None})
print_rb3_recheck("FUTURE_TRIGGER", BASE,
                  {"trigger_time": utc("14:20")})
print_rb3_recheck("LATE_APPROVAL", BASE,
                  {"approval_time": utc("14:14")})
print_rb3_recheck("WRONG_APPROVED_SCOPE", BASE,
                  {"approval_scope": {"booking-read"}})
print_rb3_recheck("CHECK_BEFORE_EXECUTION", BASE,
                  {"post_state_time": utc("14:11")})
print_rb3_recheck("PROTECTION_FAILED",
                  {**BASE, "protection_check": False}, {})
print_rb3_recheck("EARLY_VALID_TRIGGER", BASE,
                  {"trigger_time": utc("14:01")})

Predict the eight additional results and compare the RB3-RECHECK worked result. A real review would need trustworthy issuer-side approval and control receipts, source-clock semantics, the named operator, a verified pre-change baseline, linked request identifiers and separate protection and legitimate-service acceptance evidence. A passing synthetic check keeps human review open; it does not establish recovery or incident closure.

Before running: predict the twenty-two scenario lines and record which gate each case tests. Run only the offline in-memory program; then compare with the RB3 worked result in the final appendix. The printed positive status describes a bounded synthetic reversal check with human review still open, never full incident recovery.

RB3 two-role practical exercise — Frozen Reversal Packet and Independent Decision

Roles: Reversal Packet Author and Independent Reviewer. The roles are fictional classroom assignments; no participant receives real production privileges.

Objective: Test whether another learner can determine from a sealed record whether a proposed reversal has a verified baseline, justified trigger, correct approver and scope, actual execution receipt and two independent post-reversal results without asking the author for undocumented context.

Procedure

1. Choose the fixture. Use the artificial VALID_WITH_OPEN_REVIEW case or one negative RB3 variant. Do not use R1–R24 as executed rollback evidence. State the fictional date and scope.
2. Author prepares the packet. Include baseline reference and its asserted check, intervention reference and prior approval status, trigger observation, proposed inverse target and objects, required owner, approval window, receipt status, post-state protection and legitimate-service results, and review owner.
3. Freeze the packet. Save RB3-packet-v1. Do not edit it during the receiver phase. The packet is a classroom document, not a runbook for real action.
4. Reviewer works independently. Without verbal or written hints from the author, produce RB3-review-v1 that states whether the reversal may remain proposed, whether it is reported executed, whether each post-state gate has evidence, and which approval or source is still needed.
5. Reviewer identifies the next safe step. For a missing approval, request the proper owner decision. For missing execution verification, request a bounded post-state check. For a failed protection or service result, request a new owner-governed action while documenting risk.
6. Freeze the review. Compare it with the declared fixture and the Python output. Record whether a disagreement came from a missing packet field, an unsupported inference or a misunderstood decision state.
7. Retest with versioning. If needed, publish a clearly labeled RB3-packet-v2 with corrected fields and obtain a new reviewer response. Preserve the first failed answer and its reason.

Required evidence package: Both frozen versions, a source-reference table, named approving role, justified trigger, scope and window check, execution/effect status, two gate results and next human decision. The reviewer must never use silence to infer permission.

Pass: The reviewer rejects invalid or incomplete routes, accurately reports the limited positive case and identifies its unresolved human-review work. Critical fail: the reviewer authorizes real changes, promotes a proposal to execution, restores an unverified baseline, ignores a failed protection gate, or reports full incident recovery from a bounded rollback result.

Limit: Passing the two-role tabletop does not demonstrate reliability of a real rollback mechanism. It tests an independent person's understanding of a fictional control-decision packet.

Worked checkpoint

A write pause is only effective for the routes actually tested. Blocking W-UI while W-API remains unmeasured does not justify a full containment claim; a separate legitimate-service check identifies the cost to users.

Practice — Lesson 11

EX027 — Three different goals

Objective: distinguish a control action from a cause fix and a service test. Inputs: R17–R23 and sections 3.3–3.5 at 09:37 UTC. Output: a three-row table for proposed session revocation, credential reset and isolated restore test, with primary goal, prerequisite authority or evidence, observable success check, and one important limit. Do not claim that any of these actions has occurred. A score of 2 identifies the credential reset as only a candidate eradication action if credential compromise is supported, and keeps a test of a copy distinct from live-service acceptance.

Worked answer: EX027 in the final appendix.

EX028 — Changed assessment

Objective: update hypotheses without rewriting the earlier decision. Inputs: R9, R10, R12 and R17–R18. Output: a before/after 09:37 assessment note with the earlier unresolved B19 issue, what R17 adds about approved tasks, what R18 adds about B20, the hypothesis weakened, the concern strengthened, and a fact that remains unproved. Record event and availability times separately. A score of 2 does not claim the learner knew R17–R18 at the 09:20 Module 2 review or identify the human behind desk-a.

Worked answer: EX028 in the final appendix.

EX029 — Decision matrix

Objective: select a bounded containment proposal. Inputs: R18–R21 and the section 3.3 comparison. Output: a two-row matrix for session revocation and the fifteen-minute staff-write pause, each with intended effect, path it does not cover, operational cost, approving role, executing role, post-action check and rollback. Add one sentence choosing an option provisionally at 09:37. A score of 2 names R21 as tolerance for a pause, not proof that a pause was approved and executed, and verifies the service cost as well as the intended block.

Additional queue requirement for EX029: For the pause option, distinguish R21 tolerability from later approval, execution and verification. Name the front-desk queue owner, the receipt and current-state checks needed before applying an item, and a hold for a conflict or absent approval. A pending request is not an accepted booking.

RB3 extension: Additional requirement: Within the existing two-row comparison for session revocation and the fifteen-minute staff-write pause, add for each option (a) the pre-intervention state that would have to be verified, (b) an observable service-harm or protection-failure trigger that would justify seeking reversal, (c) the accountable approver of that new reversal decision, and (d) separate protection and legitimate-service checks after a reported inverse. Do not create a third row, reuse W1 probe outcomes as canonical facts or claim the option was executed at 09:37.

RB3 2-point refinement: Both original intended effects and service costs remain explicit, with R21 labeled tolerance only. The reversal is a separately approved proposal with exact target and review conditions; a hypothetical fallback does not prove recovery.

Worked answer: EX029 in the final appendix.

Continue studying — lesson 11 of at least 20

← Lesson 10 · Course lesson index · Lesson 12 →

Continue to the next authored daily lesson.

Lesson 12 — Cause-specific correction and initial recovery gates

Week 3, day 2. Work through the teaching and the worked checkpoint before attempting the assigned exercise(s). Compare your artifact with the final worked result only afterward.

3.4 Eradication requires a supported cause

Revoking a session is a containment action. It is not proof that a compromised credential, malicious browser extension, faulty training script or application bug has been removed. Before proposing eradication, the team needs to identify a cause well enough to choose a specific fix and a verification test. Examples of possible fixes include resetting a confirmed compromised credential, removing an identified malicious component, correcting a faulty application change or repairing an authorization rule. None of those causes is established in R1–R24. The case record should therefore say “cause unresolved; eradication plan pending evidence,” while continuing to check whether edits stop.

For example, suppose the session is revoked after 09:37 UTC and a new narrow query shows no later edits. That observation is compatible with at least three explanations: the revoked session caused the edits, the real source stopped independently, or the audit query missed later activity. R22, which was recorded at 09:32 UTC, cannot be used to verify a later revocation. Neither query can isolate a root cause. A proposed credential reset would become better supported if authorized identity evidence established credential misuse, but it would still require checking other sessions, tokens and recovery paths. A proposed application fix would require a reproducible software fault and a negative test demonstrating that the same unauthorized change no longer succeeds. The lesson is to match the fix and its verification to a supported mechanism.

From a plausible story to a falsifiable cause test

An eradication plan begins with a claim about how the unwanted change could occur, not with the name of a tool to run. In the fictional Harbor Library case, R17 says the training owner's listed work covered B17 and B18; R18 shows a later B20 edit associated with desk-a. That discrepancy deserves investigation, but the account label alone cannot identify the person, device or mechanism. R22's earlier quiet query, R23's backup-job status and R24's unattributed exposure concern do not close the causal gap. NIST SP 800-61 Rev. 3 places incident analysis, mitigation and learning within the wider response process. The following test design is OpenCyber Academy's classroom adaptation; it is not a finding about an actual service.

Write each candidate explanation so that a different observation could weaken it. A useful hypothesis record has five fields: the mechanism, current supporting evidence, a competing explanation, the least invasive authorized discriminator and a result that would change the decision. The discriminator should point to a defined source, owner, event interval and collection limit. “Look at more logs” lacks a falsifiable question. “Ask the identity custodian for session issuance and revocation events for desk-a around the B20 edit, with event IDs and a declared time window” can be answered or marked unavailable. Even a matching session event may identify a credential path without establishing who held the credential.

These are competing tests, not three diagnoses. A negative check is meaningful only inside its coverage. If an identity export omits older sessions, it cannot exclude older credential use; if an isolated application test exercises only the current build, it cannot exclude a fault in the earlier build. Record the source version, query interval, filters, clock assumptions and missing fields alongside the result. If a control change has already been made, preserve relevant pre-change evidence and note that a later failed reproduction may reflect the new control rather than the original behavior. A beginner can document these limits without touching a production system.

Build a cause-to-fix contract

For each supported mechanism, link evidence → authorized change → pre-change baseline → post-change test → monitoring → rollback or re-open trigger. The arrows express a decision sequence, not proof that a change succeeds. Containment can proceed while the mechanism remains uncertain; eradication requires a supported cause and a test tied to that cause. The incident lead may keep multiple hypotheses open and seek specialist help when no safe discriminator exists.

Consider a conditional credential-misuse branch. If authorized identity evidence later supports use of a specific compromised session or credential, an identity owner may propose revoking relevant sessions, rotating the credential and reviewing related tokens and recovery paths. Before calling the branch effective, the owner records which sessions and token types are in scope. A controlled synthetic login with the old credential should fail, a newly authorized login should work, and a bounded identity/application audit should show the attempts with usable times and IDs. If another token type or account remains untested, the decision record says so. An empty booking-edit query alone would not prove the old credential became unusable or that no other access path remains. None of these branch observations is supplied by R1–R24.

Consider instead a conditional application-fault branch. If an isolated reproduction shows an unauthorized synthetic account can change a booking through a particular server route, the application owner can design a server-side authorization correction for that route and check comparable routes. Keep a record of the failing baseline on the isolated pre-change build, then repeat the same labeled request against the corrected build. A credible pass requires a denied response, unchanged authoritative booking state and an audit event tied to the request. Also test that an authorized synthetic account can still carry out its intended work; a fix that disables all booking writes may stop the symptom while failing the service requirement. A rollback trigger is a later unauthorized state change or material service failure during the agreed observation window. W1 in section 3.3 illustrates why route coverage matters, but it is a separate write-pause fixture and is not evidence that Harbor Library had this application fault.

The workflow branch is different. If a later authoritative record establishes that B20 was approved but missing from R17's supplied task list, the team should correct the approval-to-execution workflow and its audit trail. It should also revise the incident classification. It would be unsound to describe a credential reset as eradication of an attacker that the evidence never established. If the missing record cannot be obtained, classify this explanation as unresolved rather than treating it as confirmed benign activity.

The decision owner should sign a short verification contract before any cause-specific change: (1) the exact mechanism claimed; (2) the evidence IDs and their limits; (3) the system and route to change; (4) the approving and executing roles; (5) the expected positive and negative synthetic tests; (6) the audit and state evidence to retain; (7) a bounded monitoring interval; and (8) the condition that reopens the case. A post-change test that was never run is recorded as planned, not passed. If the mechanism remains uncertain, the honest status is “containment under review; eradication not established.” Recovery may still be considered under temporary safeguards, but section 3.5 requires separate service-acceptance evidence.

Rebuilds, account deletion and broad purges are high-impact actions. They may destroy evidence or create recovery problems. They need the relevant owners, scope and change controls. A beginner should avoid treating “wipe and restore” as a synonym for responsible response.

3.5 Verify recovery, not merely activity

The recovery lead needs more than R23's successful backup status. A restore claim should name the copy, time, environment, expected contents and test result. NIST SP 800-34 Rev. 1, updated edition discusses contingency planning and recovery priorities; this classroom example adapts that idea to a small booking service. A safe test uses an authorized isolated copy, not the live database during an unresolved event.

For Harbor Library, a proposed return-to-service checklist is:

1. Compare the approved booking source with R20 and identify which of B17–B20 need owner-reviewed correction. Do not overwrite bookings solely from a patron report.
2. Test that authorized staff can view and create a new synthetic booking in an isolated environment, while an unauthorized test identity cannot edit one.
3. Confirm that a narrow audit query records the test change with a usable timestamp and actor label.
4. Ask the service owner to accept any remaining manual-queue entries and confirm that patrons have a consistent schedule.
5. Set a short monitoring window and an explicit rollback trigger if unexplained edits resume.

If the cause is unresolved, the lead may allow limited service with temporary safeguards and close monitoring, or may keep a restricted mode. The record must state that residual uncertainty. “The page loads” is not evidence that bookings are correct or that unauthorized edits have stopped.

Recovery objectives are promises to test, not facts supplied by a backup status

The service owner may define a recovery time objective (RTO) for how long a service can remain unavailable and a recovery point objective (RPO) for how far back its data can acceptably be restored. NIST SP 800-34 Rev. 1, section 3.2 distinguishes these objectives in contingency planning. OpenCyber Academy applies them here as planning questions for a fictional small service; a particular organization's definitions and approved targets must come from its own business-impact analysis.

As a hypothetical planning exercise, imagine the library says room booking should be usable within two hours of an interruption and no more than thirty minutes of accepted booking changes may be lost. Those are targets, not demonstrated performance. If the only usable copy were made at midnight and an interruption occurred at 09:45 UTC, that copy alone could omit more than nine hours of changes. It could not demonstrate a thirty-minute recovery point without another protected source or a reconciliation method. The actual R7 and R23 records do not provide a backup timestamp, a restore duration or a validated final schedule. They therefore establish neither objective. A successful restore test also needs the correct data and acceptable access controls; speed alone is insufficient.

Keep technical restoration and service acceptance separate. The recovery lead can report that a controlled copy starts and passes defined integrity checks. The service owner must still decide whether bookings are correct, manual-queue requests are reconciled and patrons can use the service with known limitations. The privacy contact reviews any newly supported exposure concern; R24 does not close that question. NIST SP 800-184 emphasizes planning, testing and improving recovery. The tests below are a classroom adaptation, not a claim that a backup, a single successful login or a quiet dashboard proves recovery.

3.6 Guided reconciliation and return-to-service lab

This is a paper or local-text simulation, with no live database, account or customer data. First choose and justify either proposed containment option from section 3.3. Then use the new exercise-only cards below. They are introduced to test recovery reasoning; they were not present in R1–R24 and must not be retroactively cited as evidence available to the 09:37 decision.

Card P1 — owner-approved target state at 09:40 UTC. For this lab, the fictional service owner confirms that the planned training changes to B17 and B18 may remain, while B19 and B20 should match the listed room assignments. This is a new fictional approval for the exercise, not an inference from R17.

Card V1 — isolated-copy trial of a proposed owner-approved correction. A synthetic staff test account can create a new test booking, and an unauthorized synthetic account is denied the same edit. The audit record for the authorized test includes a UTC time but has a blank actor identifier. These are supplied observations for the lab, not evidence that the production correction ran and not a test to run against any real service.

Step 1 — Reconcile state. Compare P1 and C1 row by row. List each mismatch and identify the owner who may approve a correction. Do not silently overwrite C1 or assume B17/B18 are wrong merely because they changed earlier; P1 explicitly approves their target states. Write a narrow proposed correction for mismatched bookings and a way to verify the resulting state.

Step 2 — Test access and observability. Use V1 to mark three claims as supported, contradicted or still untested: authorized staff can create a test booking; an unauthorized test identity cannot edit; the audit identifies the actor for every change. A successful application test does not repair missing audit attribution. Record whether full return-to-service acceptance is justified.

Step 3 — Account for the chosen branch. For session revocation, state that the old session's removal must be verified but cannot prove all access paths are closed. For the write pause, account for the two manual-queue requests in Branch P and obtain service-owner sign-off before resuming writes. In both cases identify one monitoring window and one observable rollback or escalation trigger.

Step 4 — Write an acceptance record. State what was verified, which claim failed, what remains unknown about the cause, who owns the unresolved audit defect, and the next review time. If you propose limited service despite the failed audit claim, specify the temporary compensating observation and the approving owner. “Recovery complete” is not an acceptable label while the target booking states or audit criteria remain unresolved.

Expected evidence: a four-row reconciliation table, a three-claim test assessment, a branch-specific queue or identity note, and a signed-off proposal for restricted or restored operation. Assessment rubric: one point each for correct mismatch identification, justified correction authority, accurate V1 interpretation, branch-specific service cost, and an honest acceptance state with an owner and review trigger. A five-point response is coherent for this lab; it is not a production recovery certificate. The worked result follows EX038 in the final answer section.

3.7 Contained lab: validate a recovery comparison before proposing changes

The paper comparison in section 3.6 assumes that every booking ID appears once, that the owner-approved target and the current copy use the same fields, and that the copy can be traced to a particular point in time. Those assumptions are easy to lose in a spreadsheet or an improvised script. A duplicate ID might cause one row to silently replace another in a dictionary; a missing ID might disappear from an inner join; an extra booking could be mistaken for an approved correction. This lab turns those cases into explicit failures or review items before anyone considers changing a booking.

The objective is to produce a deterministic read-only diff of the four synthetic P1 and C1 rows. Use only the Python standard library and in-memory text; the script opens no file, connects to no service and does not edit any record. P1 and C1 are released after the 09:37 containment decision in the simulation. Their values can inform a later recovery proposal, never the earlier decision. NIST SP 800-184 recommends planned recovery tests and improvement; the CSV schema, validation rules and output below are OpenCyber Academy's original classroom example, not a NIST procedure.

Inputs: two miniature CSV strings with exactly booking_id,state columns. The target string transcribes the owner-approved P1 values; current transcribes C1. In an actual authorized recovery, the team would record the approver, copy identifier, source time, collection method and access restriction before trusting a comparison. Those provenance fields are not supplied by this toy script and must not be fabricated. The state values are short fictional labels, not patron information.

Step 1 — Read and validate. Copy both parts of Listing 3.1, in order, into one local scratch Python 3 session you control. parse_state rejects a changed header, a blank or extra field, and a repeated booking ID. A failed input must stop the comparison; using the last duplicate row would make the output depend on input order. The program uses csv.DictReader so that the field names, not column position alone, drive interpretation.

Step 2 — Compare without mutation. compare walks the union of booking IDs. It reports MATCH, MISMATCH, MISSING_CURRENT or UNEXPECTED_CURRENT. The two latter categories are built into the method even though the ordinary four-row fixture has neither. An ID absent from C1 is not safely interpreted as “deleted” without checking whether the export filter or collection failed. An extra ID may be an unrelated legitimate booking; it must be reviewed, not overwritten automatically.

Step 3 — Challenge the input. The final line appends a second B20 row to the current fixture. The parser should reject it before a diff is printed for that variant. This is a deliberate malformed-input test, separate from the valid P1/C1 comparison. It demonstrates a data-quality failure rather than a new event in the Harbor Library case.

Listing 3.1, part 1 of 2 — fixture and validated parser. The next block completes the same program.

import csv
from io import StringIO

TARGET = """booking_id,state
B17,Room 4
B18,Cancelled
B19,Room 1
B20,Room 3
"""

CURRENT = """booking_id,state
B17,Room 4
B18,Cancelled
B19,Room 3
B20,Room 1
"""


def parse_state(label, source):
    reader = csv.DictReader(StringIO(source))
    if reader.fieldnames != ["booking_id", "state"]:
        raise ValueError(f"{label}: unexpected CSV header")
    rows = {}
    for row in reader:
        if None in row or not row["booking_id"] or not row["state"]:
            raise ValueError(f"{label}: incomplete or extra field")
        booking_id = row["booking_id"].strip()
        state = row["state"].strip()
        if not booking_id or not state:
            raise ValueError(f"{label}: blank value")
        if booking_id in rows:
            raise ValueError(f"{label}: duplicate booking ID {booking_id}")
        rows[booking_id] = state
    return rows

Listing 3.1, part 2 of 2 — comparison and run. Append this block immediately after part 1 before executing.

def compare(target, current):
    counts = {"MATCH": 0, "MISMATCH": 0, "MISSING_CURRENT": 0,
              "UNEXPECTED_CURRENT": 0}
    for booking_id in sorted(set(target) | set(current)):
        if booking_id not in current:
            status = "MISSING_CURRENT"
        elif booking_id not in target:
            status = "UNEXPECTED_CURRENT"
        elif current[booking_id] == target[booking_id]:
            status = "MATCH"
        else:
            status = "MISMATCH"
        counts[status] += 1
        print(f"{booking_id}: {status} target={target.get(booking_id, '-')}",
              f"current={current.get(booking_id, '-')}")
    print("counts:", ", ".join(f"{name}={count}" for name, count in counts.items()))


target = parse_state("target", TARGET)
current = parse_state("current", CURRENT)
compare(target, current)
try:
    parse_state("current", CURRENT + "B20,Room 1\n")
except ValueError as error:
    print("rejected variant:", error)

Supplemental frame check for the declared four-ID P1 exercise

The read-only Listing 3.1 reports differences in supplied rows. It cannot detect a booking ID absent from both files: removing B19 from P1 and C1 yields MATCH=2, MISMATCH=1 with zero missing-current rows; comparing two header-only files produces four zero counters. Neither output establishes that the approved four-booking test frame was present. Keep the original listing and its six-line transcript as a demonstration of a row comparison, then run the following standalone frame check. Its expected IDs are an independently declared premise of this fictional P1 exercise, not a method for discovering every real booking.

import csv
from io import StringIO

EXPECTED_IDS = frozenset({'B17', 'B18', 'B19', 'B20'})
P1 = '''booking_id,state
B17,Room 4
B18,Cancelled
B19,Room 1
B20,Room 3
'''
C1 = '''booking_id,state
B17,Room 4
B18,Cancelled
B19,Room 3
B20,Room 1
'''

def read_csv(source):
    reader = csv.DictReader(StringIO(source))
    if reader.fieldnames != ['booking_id', 'state']:
        raise ValueError('bad header')
    result = {}
    for row in reader:
        if None in row or not row['booking_id'] or not row['state']:
            raise ValueError('invalid field count or empty field')
        key, value = row['booking_id'].strip(), row['state'].strip()
        if not key or not value or key in result:
            raise ValueError('blank value or duplicate booking ID')
        result[key] = value
    return result

def assess_frame(target_csv, current_csv):
    target, current = read_csv(target_csv), read_csv(current_csv)
    missing_target = sorted(EXPECTED_IDS - target.keys())
    extra_target = sorted(target.keys() - EXPECTED_IDS)
    if missing_target or extra_target:
        return (f'REJECT_TARGET_FRAME missing={",".join(missing_target) or "-"} '
                f'extra={",".join(extra_target) or "-"}')
    missing_current = sorted(EXPECTED_IDS - current.keys())
    extra_current = sorted(current.keys() - EXPECTED_IDS)
    matches = sum(target[key] == current[key] for key in EXPECTED_IDS & current.keys())
    mismatches = sum(target[key] != current[key] for key in EXPECTED_IDS & current.keys())
    state = 'REVIEW_CURRENT_FRAME' if missing_current or extra_current else 'FRAME_COMPLETE_FOR_FOUR_IDS'
    return (f'{state} MATCH={matches} MISMATCH={mismatches} '
            f'MISSING_CURRENT={len(missing_current)} UNEXPECTED_CURRENT={len(extra_current)}')

CASES = [
    ('BASE', P1, C1),
    ('BOTH_OMIT_B19', P1.replace('B19,Room 1\n', ''), C1.replace('B19,Room 3\n', '')),
    ('HEADER_ONLY', 'booking_id,state\n', 'booking_id,state\n'),
    ('MISSING_CURRENT_B20', P1, C1.replace('B20,Room 1\n', '')),
    ('EXTRA_CURRENT_B21', P1, C1 + 'B21,Room 2\n'),
]
for label, expected, observed in CASES:
    print(f'{label}: {assess_frame(expected, observed)}')
assert assess_frame(P1, C1).startswith('FRAME_COMPLETE_FOR_FOUR_IDS')

Compare its five-line result with the supplemental worked output. A complete four-ID frame still contains B19/B20 mismatches. A missing or extra current row needs review, not an automatic edit; actual target approval, copy provenance, authorized correction and a new post-change comparison remain external to the script.

Expected evidence: four printed booking statuses, a count summary with two matches and two mismatches, and a separate rejected-duplicate message. Save the output as a classroom note along with the program text; record whether it was run and in which local Python version. The worked output appears in the final answer section. The B19/B20 differences are proposed correction candidates, not permission to edit. The service owner must confirm P1's authority and approve a bounded correction; an operator would then act in an authorized environment and a reviewer would compare a new state copy. C1 is only one point-in-time snapshot. A matching diff after a later correction would still need access and audit checks, including the failed actor field in V1.

Assessment: award one point each for (1) rejecting a duplicate instead of silently choosing a row, (2) identifying B19/B20 as mismatches while retaining B17/B18 as matches, (3) distinguishing missing and unexpected IDs in the method, (4) recording that target approval and copy provenance are external to the script, and (5) proposing an authorized, newly verified correction without claiming it occurred. A passing 5/5 demonstrates sound use of this small comparison, not recovery of a real service. If Python is unavailable, trace the four iterations and duplicate check on paper; the expected statuses and limits are the same.

Worked checkpoint

A suspicious account and an application fault imply different corrections. The response lead can compare a narrow test for each cause, name approval and reversal conditions, and keep eradication unclaimed while the discriminator is missing.

Practice — Lesson 12

EX030 — Risk of observation

Objective: make non-intervention an explicit, reviewable choice. Inputs: R18, R21–R22 and the 09:37 checkpoint. Output: an observation decision card with approver, ten-minute deadline, proposed audit source and interval, query-coverage limit, verified trigger for early intervention, and accepted harm if another edit occurs. State what happens if logging is unavailable or the query is incomplete. A score of 2 does not treat R22's earlier narrow zero result as evidence that the next ten minutes will be quiet.

Worked answer: EX030 in the final appendix.

EX031 — Evidence before change

Objective: preserve enough context without turning preservation into a reason for unbounded delay. Inputs: R9–R10, R13, R15 and R17–R20, all available within R1–R24 at the 09:37 checkpoint; use only the values supplied. R15 states a duration but not its deletion-clock anchor or an exact loss time. Output: a four-row preservation request naming the source/item, bounded purpose, custodian or approval, and what the item cannot establish. Add a decision rule for when verified ongoing harm justifies an approved containment action before all four requests finish; record any omitted item and reason. A score of 2 calls the records requests or supplied summaries unless a new collection actually occurred and avoids bulk patron-data copying.

Worked answer: EX031 in the final appendix.

EX032 — Avoid premature eradication

Objective: repair an unsupported closure claim by making two causes testable. Inputs: R17–R24 and sections 3.3–3.4. Output: (1) a three-sentence correction stating what a proposed session revocation is intended to contain, which actor/cause/path questions remain, and why eradication is not established; (2) a two-row hypothesis card for credential/session misuse and an application authorization fault, each with supplied support, an authorized discriminator, a result that would weaken it and a conditional fix-verification pair; and (3) one sentence explaining why R22, recorded before 09:37, cannot verify the later proposal. Keep planned tests separate from supplied observations. A score of 2 names at least one positive and one negative synthetic test for a proposed fix, plus a scope or provenance limit; it does not assert that revocation, root-cause testing or a fix happened.

Worked answer: EX032 in the final appendix.

Continue studying — lesson 12 of at least 20

← Lesson 11 · Course lesson index · Lesson 13 →

Continue to the next authored daily lesson.

Lesson 13 — Backup provenance, freshness and recovery objectives

Week 3, day 3. Work through the teaching and the worked checkpoint before attempting the assigned exercise(s). Compare your artifact with the final worked result only afterward.

3.8 Guided backup-candidate and isolated-restore lab

Section 3.5 introduced recovery objectives; section 3.6 compared booking states; section 3.7 checked whether the comparison itself silently discarded rows. A recovery lead must join those three kinds of evidence before proposing a return to service. The most recent copy can contain the very error being investigated. An older copy can appear correct for the four displayed bookings while omitting legitimate work. A valid checksum can show that a particular copy matches a recorded digest without showing that its source was trustworthy or that its contents meet the service owner's target. NIST SP 800-184 provides recovery-planning and testing guidance; the three-candidate gate below is an original, deliberately small classroom method.

This is an optional exercise-only replay, introduced after P1/C1 and V1. It does not add an event to R1–R24, does not describe a real Harbor Library backup, and cannot be used to revise the 09:37 containment choice. In this replay, imagine a service interruption at 09:45 UTC and a candidate review at 09:46 UTC. The service owner's illustrative targets from section 3.5 are a thirty-minute RPO and a two-hour RTO. Neither target was established by R7 or R23. The candidate descriptions below are supplied solely so a learner can practice deciding what still needs verification. The exercise assumes an approved isolated environment and permission to inspect only synthetic booking IDs; it authorizes no production restore.

Card K1 — three proposed comparison copies. A snapshot timestamp is the claimed data point. An isolated import duration is a measured classroom trial, not the duration of restoring the production service. The checksum comparison is against a digest recorded with that same candidate; the exercise does not establish independent authenticity of the digest or the snapshot source.

The data-point ages relative to the hypothetical 09:45 interruption are 45, 25 and 7 minutes respectively. The thirty-minute RPO would screen out K-A as a stand-alone copy if there were no verified way to recover legitimate changes after 09:00. K-B is within the time target, but its digest mismatch is a stop-and-investigate result: a responder cannot replace the recorded digest with one from the observed bytes and call the copy verified. K-C is fresh enough by the timestamp alone, yet its B19/B20 state conflicts with the later owner-approved P1 target. The matching digest does not cure that logical error. These observations do not prove that K-A is clean throughout, that K-B is malicious, or that K-C is unusable for every limited purpose.

Four separate gates before a restore proposal

1. Provenance and permission. Identify the snapshot identifier, originating system, claimed creation time and clock basis, capture method, recorded digest, custody of the copy, allowed test environment and owner who approved the test. K1 omits most of these details. A custodian should supply them in an actual case; a learner should leave the fields open rather than infer them from the candidate label. Preserve the original candidate and test a controlled working copy.
2. Copy integrity and usability. Compare bytes with the previously recorded digest, then attempt a bounded isolated import with error logging and a repeatable test. A match supports only a limited byte-integrity claim; a successful import shows that the tested platform read the copy under the trial conditions. K-B fails the first check and has no second check. Neither K-A's nor K-C's successful import proves that the complete database, dependencies or access controls work.
3. Data correctness and loss. Compare the isolated booking state with an owner-approved target and account for legitimate transactions between the snapshot and interruption. P1 approves only B17–B20. K-A matches those four rows but may lack later legitimate bookings; K-C contains the known B19/B20 mismatches. A real reconciliation needs a source for intervening work, its provenance, a conflict rule, and an owner who accepts any unrecoverable gap. If that evidence does not exist, the team reports an unknown loss bound, not “zero loss.”
4. Service acceptance and control. In a safe test scope, verify authorized and unauthorized actions, actor-labelled audit records, manual-queue reconciliation, dependencies, user-facing behavior and a monitoring/rollback condition. V1 supplies two limited access observations and one failed audit-attribution observation. It does not become a pass because a different copy imported quickly. The service owner and incident lead must decide whether restricted operation with a documented compensating check is acceptable; a full-return claim requires the failed control and other acceptance criteria to be resolved.

These gates can fail independently. A candidate can pass an integrity check and fail the owner's data comparison; another can match four rows and fail freshness; a third can be fresh but have an unverified copy. Writing one label such as “good backup” hides the decision. Use PASS, FAIL or UNKNOWN per gate, with the supplied observation beside it. UNKNOWN is not an implied pass. Where a source cannot be trusted, the correct next step is to seek a more reliable copy or independent corroboration, not to force the matrix to a single winner.

An RPO calculation is only a screening calculation

For this replay, subtract the claimed snapshot time from 09:45: K-A is 45 minutes old; K-B is 25 minutes; K-C is 7 minutes. That arithmetic tests the age of the stated recovery point. It does not measure the number of valid bookings lost, the effect of a write queue, whether a snapshot is internally consistent, or how far a restored application would lag once other dependencies are rebuilt. If the clock or source metadata is wrong, even the age calculation is only conditional. An RPO pass needs an acceptable recovered data point under the service owner's rules, not merely a recent filename.

Likewise, 11 or 14 minutes of isolated import is not an RTO result. The two-hour target concerns usable service after the interruption. A meaningful elapsed time would need a defined start and stop, a tested platform, data and access checks, downstream dependencies, and an acceptance decision. In this fixture, no production restart or owner acceptance occurred. Record the isolated times as component measurements and leave the service-level recovery time unmeasured. This avoids an attractive but invalid equation: import duration = outage duration.

Practice: write a candidate decision card

Create a four-row gate matrix for each candidate, using the gate names above. For each row, state one K1/P1/V1 observation and the reason the result is PASS, FAIL or UNKNOWN. Then write a six-line recommendation: (1) the 09:46 review time and status as proposal; (2) whether any candidate independently supports both data-point and correctness claims; (3) which copy, if any, merits a further isolated trial and under whose approval; (4) what records are needed to reconstruct legitimate changes after its snapshot; (5) how the failed V1 actor field and unresolved cause affect acceptance; and (6) the next review trigger and named owner. Do not combine K1 with the canonical case history as though these candidate tests had actually happened.

For a variation, imagine that the only independent transaction journal also contains a gap from 09:10 to 09:18 UTC. Explain why a “replay all missing transactions” plan is no longer verified. Identify the narrowest additional source or service-owner decision that could bound the missing interval. This inject belongs to the exercise only. Removing it returns you to the K1 facts; do not add it to R1–R24 or assert a real journal gap.

Expected submission: three candidate matrices, the six-line decision card and a two-sentence variation response. Assessment rubric (six points): one point each for correct age arithmetic; treating K-B's mismatch as a failed integrity gate; distinguishing K-A's four-row match from complete or timely recovery; identifying K-C's B19/B20 conflict; keeping isolated import time separate from RTO and V1's audit failure open; and proposing an authorized, reviewable next test without claiming a completed production restore. A 6/6 answer is coherent for this fixture only. The model result follows the Module 3 source-reading answer in the final appendix.

Return to service is a separate decision

A team can reduce further edits while leaving the service unusable or its booking data wrong. Equally, a restored interface may allow unauthorized changes or fail to produce an attributable audit. The return-to-service owner therefore needs a paired gate: acceptable protective behavior and acceptable legitimate business behavior, plus correct state and sufficient observability. A failed mandatory gate means the full acceptance claim is blocked.

Rollback is not simply "reverse the command." It needs a named owner, scope, the state or configuration to restore, the treatment of queued work, and tests for both renewed user service and restored security. If a broad pause must remain temporarily, the service owner should record how manual requests are received, protected from duplicates, later reconciled and reviewed. A risk exception is explicit, time-bounded and approved; it must not be hidden under the label "recovered."

The last exercise of the chapter should therefore leave four distinct artifacts: a response decision with approval state, a causal hypothesis and falsifier, an isolated recovery test request with gate results, and an owner-reviewed service acceptance decision. No one artifact alone proves the others.

3.8a Measure recovery time and the recovered data point

Learning objectives and instructional fit

After this supplement, a new incident-response learner should be able to explain which two events bound an outage-to-service-acceptance measurement, calculate elapsed time from those events when trustworthy timestamps exist, and identify why a short installation or import time is not the same measurement. The learner should also screen snapshot age against a hypothetical recovery-point target without presenting age as a count or duration of actually lost business transactions.

The earlier Chapter 03 teaches that R23's backup-job success does not establish recovery readiness. Section 3.5 introduces recovery time and point objectives; Section 3.8 uses K1 to compare candidate timestamps, copy integrity and booking correctness. Those are necessary foundations. The distinct next step is to design a measurement record from interruption through recovery testing and service-owner acceptance, so that a responder can say precisely whether an RTO result was measured, merely targeted, or impossible to determine.

The method below uses invented small records and standard-library Python. It deliberately avoids repeating the K1 candidates, using P1's four booking values as synthetic output, simulating a live write pause, or replacing the chapter's separate access-and-audit tests. The aim is to improve reasoning from the evidence the learner actually possesses.

RTE3.1 Define the unit of service that starts and stops the clock

A statement such as "the system recovered in twelve minutes" is uninterpretable until it identifies which system function, the starting event, the stopping event and the source of both times. A server process might restart in twelve minutes while a customer-facing booking workflow remains unavailable. A database might become queryable while staff writes are still blocked. Neither event necessarily proves that the library resumed its accepted reservation service.

For this classroom, define the measured unit as the owner's approved room-booking service outcome for the scoped exercise. The measurement starts when that outcome is first documented as interrupted under the agreed exercise rule. It stops only when the scoped functional, booking-data, access and audit acceptance conditions have passed and the designated service owner records an acceptance decision. The owner's decision may include a restricted-mode exception, but the claimed scope must then say restricted, not unrestricted recovery.

This is an operational measurement contract chosen for RTE3, not a universal formula required by NIST. A different organization may define outage onset, business impact and RTO scope differently in its business impact analysis. NIST SP 800-34 Rev. 1 provides official contingency-planning context, and the NIST RTO glossary ties recovery time objectives to the effect of recovery duration on mission or business processes.

The interval only answers an honest question if both endpoints apply to the same declared service scope. A start time for staff booking writes and an end time for the home page are mismatched endpoints. A measured end for one isolated test copy does not prove a live service accepted production requests. Every measurement card should therefore name the function and whether the environment is an isolated training fixture, an authorized rehearsal or a real system under appropriate controls.

RTE3.2 An RTO target is not an observed recovery duration

An objective expresses how quickly a function should recover under an owner-approved plan. A measurement reports how long an identified event actually took under a specified method. A pass/fail comparison requires both the approved objective and enough evidence for the corresponding measurement. It is improper to report "RTO achieved" merely because someone proposed a target or because a restore command ended within that target.

The current book uses a hypothetical two-hour RTO for a planning example. RTE3 retains the same illustrative magnitude: 120 minutes. The classroom accepts it only as a fixture input. It is not established that Harbor Library's real or canonical fictional service owner adopted that target. R7 and R23 supply no actual service-interruption timestamp, final accepted-service time or measured restore duration.

To make measurement defensible, use at least three distinct time concepts. Interruption time denotes when the named service outcome became unavailable. Technical-ready time records when a restricted copy or component can perform a bounded technical test. Service-accepted time records the owner's later decision after the required business and security checks. Detection and recovery-start times are additional useful milestones but are not substitutes for the two contract endpoints.

If the technical environment starts at 10:20 but the service owner accepts the complete scoped service only at 11:10, the recovery interval does not end at 10:20 under RTE3's contract. Those fifty minutes may contain evidence reviews, identity tests, corrections, queue reconciliation and owner decision-making. Ignoring them makes a technical component look faster than the service actually became acceptable to use.

RTE3.3 Construct a milestone ledger with traceable meanings

A recovery measurement ledger should capture what each timestamp means and what produced it, not simply a sequence of clock readings. The following columns form a useful minimum:

A timestamp should be stored together with its timezone, source and uncertainty. RTE3 uses one artificial calendar date and explicit UTC offset in Python solely to avoid ambiguity. In a real collection, time from different systems may need independent clock and ingestion validation. Sorting timestamp strings is not enough to establish that each source observed the same event boundary.

The ledger also needs a status, such as planned, performed, verified, failed or unavailable, for every milestone. It is inaccurate to fill a planned 11:10 acceptance slot with a value and later report it as measured if the owner never accepted the service. Preserve the original planned time and mark the actual receipt missing.

RTE3.4 What counts as an acceptable stopping event?

The end of the recovery interval must be tied to the declared outcome. For a fictional booking service, plausible acceptance gates include:

1. Data: the restored or reconciled state matches the relevant owner-approved expected objects, with explicitly bounded coverage.
2. Authorized operation: a permitted synthetic staff booking can succeed in the authorized test scope.
3. Denied operation: a prohibited synthetic edit is rejected without an unintended state change.
4. Audit: the permitted action produces the expected actor and event time within the tested route.
5. Owner decision: the accountable service owner records acceptance for the tested operating mode and notes residual restrictions.

These gates are not a substitute for real organization-specific criteria. They are intentionally separate because passing one cannot repair another. The existing V1 fixture reports two limited access outcomes and a blank actor field; that failed audit attribution criterion would prevent an unrestricted acceptance claim under the relevant later exercise. RTE3 does not change V1's outcome or pretend that any new booking correction was executed.

Owner acceptance itself should state its boundary. A service may be conditionally accepted for restricted use while a different function remains blocked, but the resulting duration can only be attributed to that restricted service definition. Labeling such a measured time as "full booking service restored" would erase the material exception.

An observed technical-ready time is valuable as a phase measurement. It can help a team understand how long platform work took. It is not, by itself, the stopping event of the declared end-to-end service-acceptance measurement.

RTE3.5 Contrastive worked example: measured service acceptance

Entirely synthetic RTE3 example, not a Harbor Library event. Imagine an isolated training service on a fictional date with the following documented teaching milestones:

The elapsed interruption-to-acceptance interval is 85 minutes: from 09:45 to 11:10. The technical-ready interval is 35 minutes, from 09:45 to 10:20. Those values answer different questions. The response detected the disruption after ten minutes, but beginning the interval at 09:55 would conceal those ten minutes of unavailable service under this contract.

Against the illustrative 120-minute target, the 85-minute simulated measurement is within target for this defined classroom service and gate set. It does not establish that Harbor Library met an approved organizational RTO, nor does it prove that every unmodeled booking function was healthy.

Suppose the recovery lead's candidate copy is labeled 09:35 and the interruption occurs at 09:45. The apparent age of the copy at interruption is ten minutes. That is a separate data-point screening calculation. Whether accepted writes after 09:35 were recovered must be evaluated from accepted-operation records and restored outcomes, not inferred from the 85-minute recovery duration.

The useful conclusion is precise: "The isolated RTE3 scenario reached owner acceptance after 85 minutes under the listed fixture gates; the copy was labeled ten minutes old at the declared interruption; and data coverage is assessed separately." It is not a claim of full operational recovery.

RTE3.6 Incorrect example: the eleven-minute import

Consider the following incorrect fictional handoff:

"The copy was imported in eleven minutes. The planned RTO is two hours, so RTO passed. The newest copy was only seven minutes old, so we lost seven minutes of data. There was no reason to wait for the service owner."

The handoff combines three unsupported conversions. Import duration is not necessarily elapsed time from interruption to accepted service. Snapshot age is a gap between two timestamps, not the amount of accepted business work permanently missing. And a successful technical activity does not replace required owner acceptance or security/audit tests.

A correct alternative would read:

"The isolated import step took eleven minutes in the separately supplied exercise. The outage start, tested dependencies, completed acceptance gates and service-owner acceptance time are not all supplied for the full service, so an interruption-to-acceptance result is not measured. The seven-minute snapshot age is a screening value under stated clock assumptions, not proof of seven minutes of lost accepted writes. Further source and owner evidence is needed before a service-level RTO or RPO conclusion."

This alternative is less dramatic but more useful. It tells the next responder exactly which evidence must be requested and which numbers remain defensible.

The distinction applies directly to K1's isolated import durations of 11 and 14 minutes. Those existing numbers describe only the classroom import-and-read trial. They do not measure a completed production outage, and they must not be reused as proof that the owner accepted correct Harbor Library booking service.

RTE3.7 A copy's age screens risk; it does not measure lost transactions

A recovery point objective (RPO) describes the point in time to which data must be recovered after an outage under the approved organizational requirement. The NIST RPO glossary links the definition to NIST SP 800-34 Rev. 1. The classroom's hypothetical RPO magnitude is 30 minutes; that is a target, not an operational result.

Snapshot age at interruption can be calculated if the relevant source/copy timestamp and interruption timestamp are known and comparable. For a copy labeled 09:35 and interruption at 09:45, the apparent age is ten minutes. A copy labeled 08:50 is apparently 55 minutes old at 09:45. This arithmetic is often a helpful initial screen for choosing further tests.

An age that exceeds 30 minutes weakens the case for relying on the older snapshot alone. It does not establish that 55 minutes of accepted business changes were actually lost. Perhaps no approved booking was accepted during part of that interval. Perhaps an independently protected transaction journal can replay later changes. Conversely, a copy only five minutes old can still contain incorrect or inconsistent state.

The correct sequence is: verify copy identity and time basis; identify the last recoverable and acceptable business state; investigate any approved journal/replay sources; compare the recovered state with an authorized ledger of accepted operations; and determine which known accepted operations remain unaccounted for.

K1 itself illustrates why copy age is not sufficient: K-B's digest mismatch blocks its integrity gate, K-C's recent state conflicts with the known approved four-booking target, and K-A's selected matches do not establish completeness of other legitimate work. RTE3 preserves those separate results rather than overwriting them.

RTE3.8 Distinguish a data-point gap from a proved loss count

Suppose a candidate snapshot is labeled 09:35 and two fictional accepted writes are recorded after that time, at 09:38 and 09:42. These teaching operation IDs are W-A and W-B. If the isolated restored state demonstrably includes both accepted effects, the controlled comparison has zero known unrecovered writes within those two listed operations.

That limited count is not a claim that the whole business lost zero data. It requires the supplied accepted-write ledger to be complete for the declared test frame. If the ledger itself is incomplete, the analyst cannot set the total loss bound to zero, even when every listed operation is present. Unknown accepted transactions might still exist outside the frame.

If W-B is absent from the verified restored state, the exercise has one known unrecovered accepted operation in its specified set. That is stronger evidence than mere snapshot age for the missing item. It still does not provide a population-wide count unless the source coverage and business effect are appropriately verified.

The learner should maintain three separate fields: claimed snapshot time, complete or incomplete accepted-operation ledger, and restored/verified operation coverage. Combining them into a single number called "minutes lost" conceals what is known and unknown.

A transaction's timestamp is not automatically the moment it became durable and accepted under every system design. In real recovery, the source owner must define its commit/acknowledgement semantics and correlation keys. Our toy IDs deliberately avoid making that assumption about Harbor Library's live reservation database.

RTE3.9 When accepted operations are missing, stop the shortcut to RPO success

The most tempting mistake in a recovery report is to declare an RPO pass because the snapshot age is below the target, even though a known accepted operation was not recovered. A data-point screen does not repair a business-state discrepancy.

For an isolated exercise, compare the declared set of accepted operation IDs with the set verified in the restored state. The set difference identifies known, specifically referenced operations requiring attention. It cannot identify operations absent from the accepted-write source, and it does not prove that an unrecovered entry was permanently destroyed rather than awaiting a later authorized replay or reconciliation.

If one known accepted write is missing, the recovery lead should record its identifier, source/time semantics, expected state, attempted replay, observed result and owner. The owner then decides whether and how the operation can be reconstructed and how the remaining risk affects service acceptance. An incident responder should not silently discard it merely to make the count zero.

This is particularly important when multiple people are working during a disruption. Manual queue receipts, application acceptance records and transaction journal entries can represent different stages of a request. A manual receipt is not necessarily an accepted booking; a successful HTTP response is not necessarily a proven durable commit. The definition of "accepted operation" needs an owner and source semantics.

RTE3 uses one simple boolean ledger_complete for the synthetic scope. In practice, proving coverage requires retention, time-window, source-health, inclusion and reconciliation checks. The boolean is a teaching premise, not an independently authenticated production attestation.

RTE3.10 What to do when the interruption timestamp is missing

Sometimes a team knows that restoration work started but cannot establish when the essential business service became unavailable. It might have an alert receipt time, a patron complaint or an operator's recollection. Those observations can help reconstruct an interval, but they are not necessarily the actual outage onset.

If the interruption time is absent, the correct result of the elapsed-time calculation is unknown. The analyst can record known milestones, for example, "first confirmed bad service observation" and "last documented good service observation," and then seek a bounded source or owner-accepted estimate. A range may be possible in a more advanced analysis; an exact duration must not be fabricated from a convenient timestamp.

Do not replace missing interruption time with detection time without saying the measurement has changed. "Minutes since alert" and "minutes the service was unavailable" are distinct metrics. A service may have been unavailable before the detector noticed it. Conversely, a detector may fire before a business disruption is established.

In the offline lab, MISSING_INTERRUPTION sets the start to None while retaining a technical-ready time and owner-acceptance receipt from the baseline. The program refuses to compute the outage-to-acceptance interval. Even the age of the snapshot at interruption becomes unknown without the reference time.

That result is a successful demonstration of evidence discipline. It is not a failed calculation in the sense of arithmetic error; it correctly declines to report an unsupported measurement.

RTE3.11 What to do when the acceptance receipt is missing or negative

A recovery team may finish all of its planned technical checks yet lack an owner decision for returning the service to use. The correct status is not "accepted by default." An absent owner receipt is a missing business decision, and the elapsed interval to owner acceptance remains unmeasured under RTE3's endpoint contract.

There is a second, stronger case: a required acceptance gate fails. Suppose a data comparison explicitly reveals an unacceptable mismatch, or the audit test lacks the required actor field. That is not merely an absent receipt. It is a supplied negative result that blocks the declared full-service acceptance. The team should record the failed criterion, its source, the responsible correction owner and the next test, without manufacturing a later acceptance timestamp.

The lab therefore distinguishes NO_OWNER_RECEIPT from ACCEPTANCE_FAILED. In the first case, the selected gates pass in the invented fixture but the final owner receipt is absent. In the second, a required gate has failed, so the service cannot be accepted under that contract despite the technical copy's availability.

An accountable owner could separately approve a restricted operating mode with explicit residual risk. That would create a different measured service scope, with its own start/stop contract and decision evidence. It does not turn a full-service failure into an unrestricted recovery pass.

No real approval, production restoration or scoped risk exception is supplied by RTE3. All owner flags are artificial program inputs.

RTE3.12 When clocks disagree, the measurement needs verification

Calculating a duration assumes that the two endpoint times can be compared. A timestamp may have the wrong timezone, a clock offset, an imprecise event definition or an ingestion delay. Merely formatting it as ISO 8601 does not establish accuracy.

A robust technical record distinguishes producer event time, collector time, review time and decision time, then documents conversions. Chapter 02 teaches those distinctions for investigation logs. In recovery measurement, the same discipline applies to outage start, copy creation and final acceptance.

RTE3 uses one fictional date and UTC-aware Python timestamps, avoiding ambiguous local clock conversion. It also includes an artificial clock_verified flag. When that flag is false, the program refuses the exact age and RTO calculations rather than invent a result. In reality, verification would require evidence about the sources and clocks; a Boolean field is only a classroom stand-in.

If the reported technical-ready time precedes the interruption under the same clock basis, or if the accepted time precedes technical readiness contrary to the defined sequence, the measurement should be marked internally inconsistent. The analyst must determine whether the records refer to different scopes or contain timing errors. The script checks some ordering conditions but is not a full forensic clock-reconstruction system.

An owner who receives an inconsistent timeline should not simply choose the timestamp that yields a passing target. Corrections need source pointers, reasoning, version history and reviewer acceptance.

RTE3.13 Use an evidence matrix rather than one recovery percentage

A single score such as "recovery 95%" can hide the facts a service owner actually needs. At the rapid level, a more useful assessment lists separate questions:

The matrix avoids turning one satisfied row into a universal pass. A service can be technically ready yet unaccepted because data is wrong. A copy may have a recent timestamp yet be unusable. An accepted restricted service may provide a useful function without meeting every normal operating criterion.

A useful handoff preserves all these statuses and names who owns the unresolved decision. The incident lead should not force a single green/red result when different functions and evidence sources remain in different states.

RTE3.14 Reproducible measurement contracts support later improvement

A recovery exercise can improve the organization only if the next rehearsal can use the same definitions. Record the version of the measurement contract, objective owner, starting event, stopping event, required gates, source timestamps, missing-data rules, and tests for an unacceptable condition. A second analyst should be able to repeat the calculation and explain why an unknown result stayed unknown.

Useful retrospective measurements include time from detection to recovery authorization, time from restoration work start to technical readiness, and time from technical readiness to owner acceptance. Those are phase measures. Their sum does not automatically equal the entire outage unless their endpoints and intervals are continuous and consistent. The end-to-end measurement still uses its separately declared start and stop.

These measures can reveal process gaps. If technical readiness occurs promptly but business acceptance takes much longer, a team might need clearer data targets, approver availability or faster audit verification. It would be premature to blame one role without examining the milestone evidence. The exercise should produce a focused improvement ticket with an owner, a test and a review date.

NIST SP 800-184 describes the role of planned recovery testing and improvement in cyber resilience. RTE3's precise timeline and fixture arithmetic are original examples built to make that principle learnable at the rapid level.

RTE3.15 Release integrity and conclusion boundaries

The canonical case's release boundary is not relaxed by this supplement. At 09:37, the learner has R1–R24 but not the later Branch S/P outcome or the separately released P1/C1 and V1 cards. R23 says the last backup job succeeded and the most recent documented restore test was thirty days earlier. It does not provide an interruption timestamp, accepted-service time or data-loss ledger. No canonical RTO or actual RPO result follows.

The K1 candidate timestamps and import durations belong only to their later classroom replay. P1/C1's four booking-state targets belong to a separate controlled comparison. V1's missing audit actor is a separate supplied test failure. MQ3's requests are invented queue rows. The RTE3 W-A/W-B events and artificial acceptance milestones may not be substituted for any of those records.

The practical success of this section is a student able to make three different statements without confusion: "The copy is this old under verified timestamp assumptions"; "These specified accepted operations were or were not found in the tested recovered state"; and "The declared service was accepted after this measured interval, if all necessary receipts exist."

If any input is missing, the conclusion must narrow rather than expand. Unknown timing is not proof of a target miss or target pass; unknown ledger completeness is not zero loss; and unverified service acceptance is not recovery completion.

RTE3 offline laboratory — measure only what the receipts support

Classification: Entirely fictional, independent RTE3 exercise. The date, interruption time, snapshot label, W-A/W-B operations, readiness milestone, gate flags and owner receipt have no relationship to R1–R24, K1, P1/C1, V1, W1, G3-DG, MQ3 or the Branch S/P cards.

Goal: Under a declared 120-minute hypothetical RTO and 30-minute hypothetical RPO, calculate snapshot age, known uncovered accepted writes in a small certified fixture ledger and elapsed interruption-to-owner-acceptance time only when justified. Negative controls must handle a missing interruption timestamp, an old copy, a failed acceptance gate, no owner receipt, a known missing accepted operation, an incomplete ledger and an unverified clock.

Environment: Python 3 standard library, no files, network connections, accounts, personal data or service operations. All rows are reset independently.

Inputs and step-by-step reproduction

The baseline's interruption is the invented 09:45 service failure on 2026-01-15 UTC; the copy is labeled 09:35, technical readiness is 10:20 and owner acceptance is recorded at 11:10. Required toy gates cover data comparison, permitted access, denied unauthorized operation and audit attribution. The declared accepted-operation set after the copy consists only of W-A and W-B; both are initially represented in the recovered set. A ledger_complete=True flag is an explicit toy premise for that selected ledger, not proof about all bookings.

Before running, predict the 85-minute accepted-service measurement; then predict which negative variants must refuse an RTO calculation or return unbounded data coverage. Copy the following into an isolated Python 3 session.

from datetime import datetime
from copy import deepcopy

RTO_TARGET_MIN = 120
RPO_TARGET_MIN = 30
DAY = "2026-01-15"

def at(clock):
    return None if clock is None else datetime.fromisoformat(
        f"{DAY}T{clock}:00+00:00"
    )

BASE = {
    "interrupted": at("09:45"),
    "copy_at": at("09:35"),
    "technical_ready": at("10:20"),
    "accepted_at": at("11:10"),
    "clock_verified": True,
    "gates": {
        "data": True,
        "authorized": True,
        "denied": True,
        "audit": True,
    },
    "owner_receipt": True,
    "accepted_writes": {"W-A", "W-B"},
    "restored_writes": {"W-A", "W-B"},
    "ledger_complete": True,
}

VARIANTS = [
    ("BASELINE_ACCEPTED", {}),
    ("MISSING_INTERRUPTION", {"interrupted": None}),
    (
        "OLD_COPY",
        {
            "copy_at": at("08:50"),
            "accepted_at": None,
            "owner_receipt": False,
            "ledger_complete": False,
            "gates": {
                "data": None, "authorized": True,
                "denied": True, "audit": True,
            },
        },
    ),
    (
        "ACCEPTANCE_FAILED",
        {
            "accepted_at": None,
            "owner_receipt": False,
            "gates": {
                "data": False, "authorized": True,
                "denied": True, "audit": True,
            },
        },
    ),
    ("NO_OWNER_RECEIPT", {"accepted_at": None, "owner_receipt": False}),
    (
        "KNOWN_MISSING_WRITE",
        {
            "accepted_at": None,
            "owner_receipt": False,
            "restored_writes": {"W-A"},
            "gates": {
                "data": False, "authorized": True,
                "denied": True, "audit": True,
            },
        },
    ),
    (
        "INCOMPLETE_LEDGER",
        {
            "accepted_at": None,
            "owner_receipt": False,
            "ledger_complete": False,
        },
    ),
    ("UNVERIFIED_CLOCK", {"clock_verified": False}),
]

def measure(row):
    i, s, t, a = (
        row[key] for key in
        ("interrupted", "copy_at", "technical_ready", "accepted_at")
    )

    if s is None or i is None or s > i or not row["clock_verified"]:
        age = "UNKNOWN"
    else:
        minutes = int((i - s).total_seconds() // 60)
        age = (
            f"{minutes}m_"
            f"{'WITHIN' if minutes <= RPO_TARGET_MIN else 'OVER'}_"
            f"{RPO_TARGET_MIN}m"
        )

    missing = row["accepted_writes"] - row["restored_writes"]
    if not row["ledger_complete"]:
        data = "UNBOUNDED"
    else:
        data = f"{len(missing)}_KNOWN_UNRECOVERED"

    if i is None:
        rto = "UNKNOWN_INTERRUPTION"
    elif not row["clock_verified"]:
        rto = "UNKNOWN_CLOCK"
    elif t is not None and t < i:
        rto = "INVALID_ORDER"
    elif any(value is not True for value in row["gates"].values()):
        rto = "NO_ACCEPTANCE_GATES"
    elif not row["owner_receipt"] or a is None:
        rto = "NO_OWNER_ACCEPTANCE"
    elif a < i or (t is not None and a < t):
        rto = "INVALID_ORDER"
    else:
        minutes = int((a - i).total_seconds() // 60)
        rto = (
            f"{minutes}m_"
            f"{'WITHIN' if minutes <= RTO_TARGET_MIN else 'OVER'}_"
            f"{RTO_TARGET_MIN}m"
        )

    return f"age={age} unrecovered={data} RTO={rto}"

for label, changes in VARIANTS:
    row = deepcopy(BASE)
    row.update(changes)
    print(label, measure(row))

assert measure(BASE) == (
    "age=10m_WITHIN_30m "
    "unrecovered=0_KNOWN_UNRECOVERED "
    "RTO=85m_WITHIN_120m"
)

Supplemental RTE3 full-service acceptance filter

The original eight-case program reproduces its stated fixture, but its RTO branch trusts gate flags without checking that the four required gates exist, the declared ledger is complete, all accepted writes were restored, or a technical-ready milestone exists. With otherwise green flags, removing W-B only from restored_writes still prints RTO=85m_WITHIN_120m; setting gates={} does the same. This is an unsafe full-service acceptance inference. An elapsed interval between supplied timestamps can be arithmetically plausible even when acceptance is unsupported. The supplemental validator below retains the original fixture and demonstrates why those cases must not receive a favorable full-service result.

from copy import deepcopy
from datetime import datetime

DAY = '2026-01-15'
RTO_TARGET_MIN, RPO_TARGET_MIN = 120, 30
REQUIRED_GATES = {'data', 'authorized', 'denied', 'audit'}

def at(clock):
    return None if clock is None else datetime.fromisoformat(f'{DAY}T{clock}:00+00:00')

BASE = {
    'interrupted': at('09:45'), 'copy_at': at('09:35'),
    'technical_ready': at('10:20'), 'accepted_at': at('11:10'),
    'clock_verified': True,
    'gates': {'data': True, 'authorized': True, 'denied': True, 'audit': True},
    'owner_receipt': True,
    'accepted_writes': {'W-A', 'W-B'}, 'restored_writes': {'W-A', 'W-B'},
    'ledger_complete': True,
}

def stricter_measure(row):
    i, s, t, a = (row[k] for k in ('interrupted', 'copy_at', 'technical_ready', 'accepted_at'))
    clocks_ok = row['clock_verified'] is True
    age = 'UNKNOWN'
    if clocks_ok and i is not None and s is not None and s <= i:
        delta = int((i-s).total_seconds() // 60)
        age = f'{delta}m_{"WITHIN" if delta <= RPO_TARGET_MIN else "OVER"}_{RPO_TARGET_MIN}m'
    missing = row['accepted_writes'] - row['restored_writes']
    coverage = 'UNBOUNDED' if row['ledger_complete'] is not True else f'{len(missing)}_KNOWN_UNRECOVERED'
    if i is None:
        outcome = 'UNKNOWN_INTERRUPTION'
    elif not clocks_ok:
        outcome = 'UNKNOWN_CLOCK'
    elif t is None:
        outcome = 'UNKNOWN_TECHNICAL_READY'
    elif t < i or (a is not None and (a < t or a < i)):
        outcome = 'INVALID_ORDER'
    elif set(row['gates']) != REQUIRED_GATES:
        outcome = 'REQUIRED_GATES_MISSING'
    elif any(row['gates'][k] is not True for k in REQUIRED_GATES):
        outcome = 'NO_ACCEPTANCE_GATES'
    elif row['ledger_complete'] is not True:
        outcome = 'DATA_COVERAGE_UNKNOWN'
    elif missing:
        outcome = 'KNOWN_UNRECOVERED_WRITES'
    elif row['owner_receipt'] is not True or a is None:
        outcome = 'NO_OWNER_ACCEPTANCE'
    else:
        minutes = int((a-i).total_seconds() // 60)
        outcome = f'{minutes}m_{"WITHIN" if minutes <= RTO_TARGET_MIN else "OVER"}_{RTO_TARGET_MIN}m'
    return f'age={age} unrecovered={coverage} RTO={outcome}'

VARIANTS = [
    ('BASELINE', {}),
    ('MISSING_WRITE_DESPITE_GREEN_GATES', {'restored_writes': {'W-A'}}),
    ('INCOMPLETE_LEDGER_DESPITE_GREEN_GATES', {'ledger_complete': False}),
    ('NO_REQUIRED_GATES', {'gates': {}}),
    ('NO_TECHNICAL_READY', {'technical_ready': None}),
    ('AUDIT_FAILED', {'gates': {**BASE['gates'], 'audit': False}}),
    ('NO_OWNER_RECEIPT', {'owner_receipt': False, 'accepted_at': None}),
    ('MISSING_INTERRUPTION', {'interrupted': None}),
    ('BAD_TIME_ORDER', {'accepted_at': at('10:19')}),
]
for label, changes in VARIANTS:
    case = deepcopy(BASE)
    case.update(changes)
    print(f'{label}: {stricter_measure(case)}')
assert stricter_measure(BASE).endswith('RTO=85m_WITHIN_120m')

Predict the nine outputs, then compare the supplemental worked result. ledger_complete and gate flags are classroom premises, not authenticated records. The favorable baseline means only that this declared fictional scope passes the toy contract; it cannot establish complete institutional data recovery, zero actual loss, or an accepted Harbor Library service. A restricted service needs a separately approved scope and receipt.

Check your eight-line transcript and reasoning in the final worked result. The code demonstrates a declared toy contract; its flags and timestamps are not authenticated recovery receipts.

The script uses artificial UTC-aware timestamps on a single fictional day and Boolean gate values. It does not collect a source, verify that an owner signed anything, reconcile a real database, ensure durable journal replay, test actual access controls, recover a live service or authenticate clock sources. It also assumes no unmodeled interval ambiguity and counts accepted IDs rather than evaluating every business effect.

Additional optional negative controls can be performed on a copied fixture: change accepted_at to precede technical_ready, or set technical_ready before interrupted. The expected status should be INVALID_ORDER when the contradictory condition is reached under the existing gates. Restore the baseline before recording the principal eight-line transcript. These are mathematical-input checks, not extra canonical incident injects.

The student should submit a matrix with scenario, start evidence, snapshot-age screening status, accepted-operation coverage, acceptance-gate status, computed elapsed duration or reason for refusal, and the next specific owner/source request. The successful measurement must state its scope, not merely the numbers.

Lab pass criteria: Reproduce all eight lines; explain why 85 minutes includes the complete fictional interruption-to-acceptance interval; identify the old-copy screening failure without inventing lost-data duration; explain why UNKNOWN_INTERRUPTION, NO_ACCEPTANCE_GATES and NO_OWNER_ACCEPTANCE are different; and keep RTE3 outside R1–R24, K1, P1/C1 and V1.

Practical activity — Build a Recovery Evidence Receipt

Learning goal: Turn a loose recovery statement into an auditable, owner-reviewable recovery measurement request.

Inputs: Use the RTE3 fixture and the pre-existing Chapter 03 objectives for context, but do not import the simulated W-A/W-B records into the canonical case.

Steps:

1. Write a one-sentence service acceptance scope, including the functions a user must perform and the operating restrictions, if any.
2. State who owns the outage-start definition and what evidence would establish the starting timestamp.
3. Name the planned technical-ready measurement and explain why it does not end the service outage under the selected contract.
4. Define four required acceptance gates: booking correctness, permitted action, prohibited action and actor/time audit result. Give each a result state and owner.
5. Name the owner-acceptance receipt and identify what happens if the owner rejects or does not respond.
6. For the data-point question, list claimed snapshot time, interruption time, post-snapshot accepted-operation ledger, restored operations, and the source that would establish completeness.
7. Calculate snapshot age separately from the accepted-service interval when both are supported. If either cannot be calculated, record the missing source and do not substitute a guess.
8. Provide a short handoff with the planned next observation, decision owner and explicit no-production-change boundary.

Expected artifact: A Markdown table headed RTE3 recovery evidence receipt, followed by one paragraph interpreting an age-only result and one paragraph interpreting a measured acceptance time. State the limitations of every figure. Do not call a copied example a completed test unless the learner executed it.

Evaluation: Full credit requires distinguishable start and stop conditions, independently supported acceptance gates, age-versus-loss separation, explicit missing-data states, source attribution and permission boundaries. A paper calculation alone does not prove a restore occurred.

Section synthesis

A recoverable copy, a short import, an owner-approved service and a valid recovery objective are distinct concepts. To measure interruption-to-service-acceptance time, define the user-facing service, verify the start event, keep intermediate readiness tests separate, require all mandatory acceptance evidence and record the owner's decision as the scoped stop event. To discuss the recovery point, screen timestamp age but separately establish what accepted business changes are recoverable, missing, unverified or outside source coverage.

RTE3 is deliberately fictional and offline. Its results do not change the canonical Harbor Library chronology, the separate K1/P1/C1/V1 evidence or the public book's completion status.

Worked checkpoint

A completed backup job gives a candidate to evaluate, not a usable service. An isolated restore must compare provenance, recovered data point, elapsed recovery time and owner-approved booking correctness before any return decision.

Practice — Lesson 13

EX033 — Recovery test

Objective: distinguish restoration planning from a measured result. Inputs: R23 and, after the later lab card is released, P1; V1 remains a separate access-and-audit trial. Output: a one-paragraph isolated-copy test plan plus a four-row B17–B20 expected-state table. Name the copy identifier and timestamp as fields to collect, actual restore start/end as fields to measure, service-owner-approved target, pass/fail/not-tested rule, and privacy boundary. A score of 2 keeps all four rows not tested until the restore is performed and never changes a live booking.

RTE3 extension: Extend the original one-paragraph isolated-copy plan with a separate, compact measurement card naming (a) the owner-defined service interruption start and source, (b) isolated restoration start and technical-ready times, (c) the booking, access and audit gate receipt fields, (d) the owner acceptance timestamp and scope, and (e) the rule for declaring elapsed time unknown. The existing B17–B20 target table remains not tested until the relevant later card and actual isolated result are supplied.

RTE3 scoring note: A plan with no actual outage or acceptance timestamps does not report achieved RTO. The learner distinguishes proposed and executed steps, remains within an authorized isolated environment and does not infer an accepted service from a copied test plan.

Worked answer: EX033 in the final appendix.

EX034 — Backup limit

Objective: test what a successful backup job can and cannot demonstrate. Inputs: R23 and the hypothetical two-hour RTO, thirty-minute RPO, midnight-only copy and 09:45 interruption in section 3.5. Output: a calculation note with 09:45 − 00:00, comparison to the thirty-minute target, the missing timing measurement for RTO, and two additional authorized checks before calling the service recoverable. Do not assert that the fictional library approved those targets or that the actual backup was made at midnight. A score of 2 shows the arithmetic, separates copy age from actual data-loss measurement and identifies booking-state and elapsed-restore tests.

RTE3 extension: Keep the existing midnight-to-09:45 arithmetic and hypothetical 30-minute RPO/120-minute RTO. Add three explicit columns: copy age screen, accepted-transaction evidence after the copy, and actual restored/verified transaction result. Ask whether any record proves a specific legitimate write was irrecoverably lost. If the accepted ledger, replay evidence or clock semantics are missing, record unknown, not "9 hours 45 minutes of data lost."

RTE3 scoring note: Correct age arithmetic must be paired with an appropriately limited inference. R23 supplies only a backup-job status and old restore-test record, not interruption, service acceptance, actual recovered point or measured data loss. An isolated import duration alone is not a service-level RTO.

Worked answer: EX034 in the final appendix.

Continue studying — lesson 13 of at least 20

← Lesson 12 · Course lesson index · Lesson 14 →

Continue to the next authored daily lesson.

Lesson 14 — Evidence-linked recovery decision records

Week 3, day 4. Work through the teaching and the worked checkpoint before attempting the assigned exercise(s). Compare your artifact with the final worked result only afterward.

3.9 Sourced case study: recovery beyond the backup copy

Harbor Library is invented. A useful real-world contrast is the British Library's own March 2024 review of the October 2023 cyber-attack. The report is the institution's retrospective account, written after investigation; it is not a raw event log, a finding about Harbor Library or a template for a one-month course to reproduce. Its exact descriptions are time-bound to the review date. Read the cited pages when using the case and avoid extending the report's conclusions to unexamined systems.

The review reports that attackers encrypted or destroyed a substantial part of the Library's server estate and that the institution had secure copies of its digital collections, yet lacked viable infrastructure on which to restore services promptly (executive summary, pp. 2–3). By December 2023, viable backup sources had been identified for collections and other data, but the report still describes restricted research services and systems needing modification, migration or replacement (pp. 2–3 and 9). A searchable catalogue returned on 15 January 2024 while other functions remained incomplete (pp. 2 and 9–10). These are reported facts about the British Library, not evidence about the fictional bookings B17–B20. The teaching inference is that copy availability, platform readiness, data validation and service acceptance are different checkpoints. A backup can exist while the environment needed to use it is missing or unsafe.

The same report describes strategic and operational crisis groups coordinating technical response, approving workarounds and communicating during disrupted website and intranet service (section 3, pp. 12–13). This supports a second teaching inference: response records must name decision owners and alternate communications paths, because a status page or ordinary internal channel may itself be unavailable. The British Library's roles and governance are specific to that institution; the incident-lead/service-owner matrix in this course is a fictional teaching arrangement, not a claim that the real Library used it.

Guided source-reading activity: write a three-row claim ledger with (1) a fact from the British Library report and its page or section, (2) one planning question it raises for the fictional booking service, and (3) one conclusion it cannot support about B17–B20. Include at least one row about restoration infrastructure and one about partial service. Then compare the source date with the time of your own course reading: a 2024 retrospective cannot by itself tell you the Library's current service state. The report also expresses uncertainty about the exact initial intrusion point; do not turn a likely entry point into a certain one. A model ledger appears in the final answer section. This supplementary source analysis is separate from EX027–EX038.

Contrast: a decision record at 09:37

Plausible but incorrect: "At 09:37 we know B20 was an attack. R21 authorizes a write pause, R22 proves the writes have stopped, and R23 proves recovery is ready. Apply the pause and close the incident after the page loads."

Each asserted upgrade is wrong. R18 reports a B20 change but no human attacker or causal mechanism. R21 says a pause is tolerable, not approved or executed. R22 is a narrow earlier query, not a post-change verification. R23 is a backup/old restore-test status, not a current service-acceptance result. A loading page is not evidence of booking correctness or auditability.

Defensible 09:37 proposal: "R17 limits the known approved task list to B17/B18 and R18 reports a further B20 edit under desk-a. The incident lead proposes a bounded current-session revocation, subject to identity-owner approval and preservation of feasible narrow sources. The identity administrator will verify execution if approved; the application and service owners will separately assess later edits and front-desk continuity. R21 supports discussing a short manual-queue fallback but gives no execution authority. Root cause, recovery readiness and personal-data exposure remain unresolved; reassess at the named next review or on a verified continuing edit."

This record does not avoid action. It prevents recommendation, approval, execution and success from becoming a single unsupported sentence.

3.10 Worked decision record and synthesis

Audit the defensible 09:37 proposal above as four separate receipts. This prevents a well-worded recommendation from being mistaken for a change that happened:

The proposal should set a 09:45 review or earlier review on a verified new unexplained edit. Root cause and personal-data exposure remain unresolved; the four rows do not claim a production action.

This chapter converts the earlier timeline into a bounded decision. Module 4 will test the whole response: intake, evidence, authority, proportionate action, recovery criteria and a review of what the team would change before another incident.

3.11 Offline laboratory: decision and recovery gates

Purpose: Validate the logical separation of proposal, approval, execution, protective verification, legitimate service continuity, data correctness and audit attribution. This is a new independent classroom branch, not a replay of W1 write-path tests, the P1/C1 booking-value comparison, V1's specific observations or K1 copy selection.

Environment: Python 3 standard library; data only in memory. No network, files, credentials or system changes. Every G3 row is a separate reset scenario, not a chronological incident timeline.

Inputs and meanings: A value of True means the corresponding fictional test card asserts that a gate passed in its limited scope; False means the card asserts a failed gate; None means no test result is supplied. The flags are not real approval receipts, collection evidence or actual restorations. protective concerns the bounded harmful operation, legitimate the required legitimate business function, data an isolated owner-approved data check and audit sufficient attribution for the tested operation. The model intentionally requires all four for its most favorable limited label.

Executable fixture

SCENARIOS = [
    ("G3-P", False, False, None, None, None, None),
    ("G3-A", True, False, None, None, None, None),
    ("G3-E", True, True, None, None, None, None),
    ("G3-C", True, True, False, True, True, True),
    ("G3-B", True, True, True, False, True, True),
    ("G3-I", True, True, True, True, None, None),
    ("G3-D", True, True, True, True, False, True),
    ("G3-U", True, True, True, True, True, False),
    ("G3-V", True, True, True, True, True, True),
    ("G3-X", False, True, True, True, True, True),
]

def gate(approved, executed, protective, legitimate, data, audit):
    if executed and not approved:
        return "INVALID_UNAPPROVED_EXECUTION"
    if not approved:
        return "PROPOSED_ONLY"
    if not executed:
        return "APPROVED_NOT_EXECUTED"
    if protective is None or legitimate is None:
        return "EXECUTED_UNVERIFIED"
    if not protective:
        return "PROTECTION_GATE_FAIL"
    if not legitimate:
        return "BUSINESS_GATE_FAIL"
    if data is None or audit is None:
        return "RECOVERY_CHECKS_PENDING"
    if not data:
        return "DATA_GATE_FAIL"
    if not audit:
        return "AUDIT_GATE_FAIL"
    return "BOUNDED_TEST_GATES_PASS"

for ident, *fields in SCENARIOS:
    print(ident, gate(*fields))

# Negative control: invalidate attribution for the favorable test card.
print(
    "NEGATIVE_AUDIT",
    gate(True, True, True, True, True, False),
)

Exact expected output — executed against the displayed fixture

G3-P PROPOSED_ONLY
G3-A APPROVED_NOT_EXECUTED
G3-E EXECUTED_UNVERIFIED
G3-C PROTECTION_GATE_FAIL
G3-B BUSINESS_GATE_FAIL
G3-I RECOVERY_CHECKS_PENDING
G3-D DATA_GATE_FAIL
G3-U AUDIT_GATE_FAIL
G3-V BOUNDED_TEST_GATES_PASS
G3-X INVALID_UNAPPROVED_EXECUTION
NEGATIVE_AUDIT AUDIT_GATE_FAIL

G3-DG adversarial follow-up: a failed gate outranks an unknown one

The first G3-DG listing reports only one state per card. Its ordering hides a known failed protection test when the service test is unknown: gate(True, True, False, None, True, True) returns EXECUTED_UNVERIFIED. Likewise, gate(True, True, True, True, False, None) returns RECOVERY_CHECKS_PENDING despite a known failed data test. These results reproduce the code's branch order, not a defensible acceptance decision. First ask whether a required test is known to have failed; only then classify the remaining unknowns. This follow-up is a separate logic-only check and leaves the eleven original lines intact.

def gate_strict(approved, executed, protective, legitimate, data, audit):
    if executed and approved is not True:
        return "INVALID_UNAPPROVED_EXECUTION"
    if approved is not True:
        return "PROPOSED_ONLY"
    if executed is not True:
        return "APPROVED_NOT_EXECUTED"
    if protective is False:
        return "PROTECTION_GATE_FAIL"
    if legitimate is False:
        return "BUSINESS_GATE_FAIL"
    if protective is not True or legitimate is not True:
        return "EXECUTED_UNVERIFIED"
    if data is False:
        return "DATA_GATE_FAIL"
    if audit is False:
        return "AUDIT_GATE_FAIL"
    if data is not True or audit is not True:
        return "RECOVERY_CHECKS_PENDING"
    return "DECLARED_GATES_PASS_NOT_AUTHENTICATED"

CASES = [
    ("ALL_FLAGS_TRUE", (True, True, True, True, True, True)),
    ("PROTECTION_FAIL_SERVICE_UNKNOWN", (True, True, False, None, True, True)),
    ("SERVICE_FAIL_PROTECTION_UNKNOWN", (True, True, None, False, True, True)),
    ("DATA_FAIL_AUDIT_UNKNOWN", (True, True, True, True, False, None)),
    ("AUDIT_FAIL_DATA_UNKNOWN", (True, True, True, True, None, False)),
    ("ALL_CHECKS_UNKNOWN", (True, True, None, None, None, None)),
    ("EXECUTED_WITHOUT_APPROVAL", (False, True, True, True, True, True)),
]
for name, flags in CASES:
    print(f"{name}: {gate_strict(*flags)}")

Supplemental output (seven lines):

ALL_FLAGS_TRUE: DECLARED_GATES_PASS_NOT_AUTHENTICATED
PROTECTION_FAIL_SERVICE_UNKNOWN: PROTECTION_GATE_FAIL
SERVICE_FAIL_PROTECTION_UNKNOWN: BUSINESS_GATE_FAIL
DATA_FAIL_AUDIT_UNKNOWN: DATA_GATE_FAIL
AUDIT_FAIL_DATA_UNKNOWN: AUDIT_GATE_FAIL
ALL_CHECKS_UNKNOWN: EXECUTED_UNVERIFIED
EXECUTED_WITHOUT_APPROVAL: INVALID_UNAPPROVED_EXECUTION

Predict which gate wins when one test is False and another is None, then compare with the seven lines. ALL_FLAGS_TRUE is an internally consistent assertion in a synthetic card; it does not authenticate an approver, operator, source, clock or test scope. A real decision requires separate receipts and a service-owner acceptance decision. Neither this fixture nor its output adds an R-series observation.

Interpretation and negative control

G3-P records only a proposal. G3-A adds a fictional approval but no execution. G3-E claims execution while omitting effectiveness checks; it cannot be described as successful containment. G3-C and G3-B show why the protective and legitimate-service tests are independently necessary. G3-I has both of those bounded passes but no recovery-data or audit result. G3-D fails data correctness. G3-U fails audit attribution. G3-V passes the declared classroom gates for that one modeled test, but does not establish root cause, complete route coverage, production authorization or service-owner acceptance. G3-X contains an internally unacceptable claim of execution without approval and is rejected even when all other flags are true.

The negative control changes only the audit flag of the favorable test to False. The output becomes AUDIT_GATE_FAIL: a missing or failed mandatory attribution check blocks the limited acceptance label even when protection, legitimate service and data appear satisfactory.

Important algorithmic limit: This code is a decision-logic demonstration, not an evidence authentication engine. Setting approved=True does not supply a real signed approval; setting executed=True does not produce an operator receipt. All gates would need independent, scope-appropriate evidence in a real authorized operation. The program also does not estimate probabilities or verify that all paths were tested.

Learner artifact: Before running, predict all eleven lines. Afterward, provide a three-column table scenario | returned state | missing or failed evidence. Add one sentence explaining why G3-V does not mean "recovery complete," and another showing how G3-D might require a new isolated copy or owner-approved reconciliation rather than an unconditional return. Keep G3 cards outside canonical R1–R24 and the later chapter branches.

Acceptance rubric (five-point lab, supplementary rather than a new numbered exercise): one point each for exact output; correct lifecycle separation; independent protective and service gates; distinct data and audit failure interpretation; and an explicit bounded-evidence/canonical-release limitation. Do not claim the classroom True inputs were verified production observations.

Worked checkpoint

An attractive restore candidate can pass integrity but fail the owner target or a bounded RPO test. The decision record should state the failed gate, a safe alternative and the next observation that could change the verdict.

Practice — Lesson 14

EX035 — Return-to-service criteria

Objective: require evidence for a proposed return to service. Inputs: P1/C1 and V1 only after their later release, plus the section 3.5 checklist. Output: three measurable acceptance rows for booking correctness, tested access behavior and audit attribution, each with expected result, current supplied result, owner and action needed. Conclude with a proposed service state, next review and one rollback trigger. A score of 2 marks B19/B20 as mismatched in C1, V1's tested access checks as limited passes, and the blank audit actor as a failure, so it does not declare unrestricted recovery.

Additional queue gate for EX035: If the later Branch P is chosen and only after it is released, add a fourth acceptance row for its two unnamed pending manual requests. The row must require a bounded manifest, owner-approved dispositions and a check of current authoritative booking state; do not borrow MQ3 outcomes, W1 or P1/C1 as the branch queue. The B19/B20 mismatches and V1 missing actor remain independent failed gates.

RTE3 extension: Retain the original booking-correctness, permitted/denied access and audit acceptance conditions and the conditional Branch P queue gate after its proper release. Add an acceptance-time evidence row: actual interruption timestamp/source, last passed mandatory gate, owner acceptance (or rejection) timestamp, scope and outcome. If P1/C1 or V1 reports a failed gate in its later isolated test, explain why no full-service recovery completion time is established by those cards.

RTE3 scoring note: Do not turn the existing V1 limited access results into a completed restore, turn a blank actor into a pass, treat a pending queue as reconciled, or report a measured RTO without comparable start/end receipts and owner-approved acceptance.

Worked answer: EX035 in the final appendix.

EX036 — Decision record

Objective: keep a response recommendation auditable before execution. Inputs: R17–R21 and the 09:37 checkpoint. Output: a six-field card covering time and status, evidence and uncertainty, chosen action and scope, approvers/executor, intended effect plus service side effect, and verification/fallback/review. Name the identity owner and incident lead for the proposed revocation; use R21 only for a possible pause fallback. A score of 2 gives an observable post-action test and says proposed, not completed.

RB3 extension: Additional requirement: Keep the original six-field card for the 09:37 proposal. In field 6 (verification/fallback/review), add a short conditional reversal subcard: last verified configuration reference to obtain, observable trigger, scope and window to request, authority needed, operator/effect receipt to require, post-reversal protection and legitimate-service checks, and the accountable human-review item. Use pending/not supplied for every rollback execution or test not contained in R17–R24.

RB3 2-point refinement: The historical action remains a proposal, the new inverse is not silently authorized, and later branch or RB3 exercise observations never enter the canonical 09:37 decision.

Worked answer: EX036 in the final appendix.

Continue studying — lesson 14 of at least 20

← Lesson 13 · Course lesson index · Lesson 15 →

Continue to the next authored daily lesson.

Lesson 15 — Queue reconciliation before return to service

Week 3, day 5. Work through the teaching and the worked checkpoint before attempting the assigned exercise(s). Compare your artifact with the final worked result only afterward.

3.12 Reconcile a manual queue before resuming writes

Why a pause ending is not proof that bookings are safe to accept again

A bounded staff-write pause can be a reasonable containment choice while the incident lead investigates unexplained booking edits. It may also create a second operational risk: new requests arrive while ordinary application writes are blocked. If staff record those requests manually, they become pending obligations, not automatically accepted reservations.

The existing Module 3 already teaches how to propose a pause, validate the configured write paths through the separate W1 laboratory, and compare authoritative booking states through P1/C1. It also says queued requests must be reconciled before approved resumption. The missing intermediate skill is determining whether each request remains valid, has already been applied, conflicts with current state, or lacks the evidence required to apply it safely.

A pause does not make every queued instruction correct. During a pause, a staff member might copy the same request twice, a patron might retry after receiving no confirmation, an appointment time might become unavailable, or someone might submit a request that was never approved. When writes are reenabled, automatically replaying everything in arrival order could create duplicate reservations, overwrite legitimate later changes, or mislead a patron into believing an unaccepted request was confirmed.

The required outcome is not "empty the queue." It is account for every queued request with a supported disposition: applied exactly within its authorized scope, recognized as a duplicate, held for approval, rejected as conflicting, or left pending with a named decision owner. These are process dispositions, not claims that a production booking was changed.

NIST SP 800-61 Revision 3 addresses incident response as part of cybersecurity risk management, including considering operational consequences. NIST SP 800-184 discusses recovery planning and tests. The queue contract below is an original Harbor Library teaching method, not a prescribed NIST process or evidence of any real library service.

Five states of a pending booking request

Before discussing a replay or service resumption, distinguish five questions about every proposed manual request.

Received: Was a request recorded, under which unique receipt ID, at what claimed time, by which authorized intake role and from which permitted channel? A row with a booking label but no trustworthy receipt might be a note or duplicate, not an actionable instruction.

Authorized: Does the request have the approval appropriate to the specific action? A permission to collect a manual queue is not necessarily permission to edit a booking. The service owner should define the action authority and any patron confirmation required by the actual process.

Current: Does the request still make sense relative to the now authoritative schedule? An old expected room/version may no longer exist. A prior room assignment is not a guarantee it remains available.

Unique in effect: Is this a second receipt of the same intended operation, a different operation with a reused deduplication key, or a separate legitimate request? Counting receipts is not the same as counting changes.

Recorded after decision: Is there a durable, auditable outcome for the attempted application or hold? A queue item should not disappear simply because an operator tried to apply it. A retry after an uncertain acknowledgement requires checking whether the original effect occurred.

A beginner-friendly queue ledger records both the original request and its evolving review state. A useful minimum schema is:

For Harbor Library, these are proposed educational fields. The canonical Branch P card does not provide any actual queue manifest. Its statement "two pending requests" cannot support claims about which rooms were requested, who submitted them or whether either can safely be applied.

Duplicate receipts are not necessarily duplicate business actions

Imagine a patron asks to move a booking. The front desk writes down the request, receives a repeated call, and writes a second note. Two receipts may describe one logical requested change.

An idempotency key is an application-defined way to recognize a repeated logical operation. The crucial condition is that the service must define which exact fields constitute the same request and what outcome follows when the key is reused. A repeat with the same key and same normalized operation may be classified as a retry; a repeat with the same key but different requested operation is a conflict demanding review, not permission to overwrite the first instruction.

This teaching technique is related to, but not guaranteed by, HTTP method semantics. RFC 9110 §9.2.2 defines idempotent methods in terms of the intended effect of repeated identical requests and cautions about automatically retrying non-idempotent methods. It does not grant a booking application's POST endpoint built-in exactly-once behavior or define Harbor Library's custom operation keys. The example below explicitly supplies its own key rules.

A key alone cannot prevent duplicates if it is stored only in temporary client memory or checked separately from the write. In a real, authorized implementation, the key-to-result record, version precondition and business-state update would need suitable consistency and atomicity guarantees. Otherwise an application failure could commit a booking while losing the recorded acknowledgement, and the next retry might apply the same business effect again.

The rapid course need not implement a distributed transaction manager. It must recognize why "the staff did not receive a confirmation" is not the same as "the write did not occur." The correct next step under that uncertainty is to verify the prior effect using an authorized source, not replay blindly.

State versions prevent silent lost updates

A manual request should identify what state it assumed. Suppose an instruction means "change booking X10 from its version 2 state to Room D." If another approved operation already advanced X10 to version 3, applying the old instruction without review could overwrite that newer decision.

A simple state-version precondition expresses the rule:

Apply the requested change only if the current authoritative version still equals the version the request expected.

If the versions differ, classify the request as stale and seek reconciliation. Stale does not automatically mean malicious, rejected forever or already fulfilled. It means the original operation cannot be assumed safe against the state now observed.

In a real concurrent database, comparing a version and updating a value in two unprotected steps would still permit a race: another operation might change the state between the check and the write. The service would need an appropriate conditional transaction or equivalent mechanism under its database consistency model. Our offline Python example runs sequentially, so it cannot prove such concurrency safety.

This also explains why an older backup is not automatically a valid replay baseline. A recovered booking state must be reconciled with accepted work and current authority before pending manual instructions can be applied. The original P1/C1 and K1 laboratories examine different parts of that recovery question; MQ3 does not supply their evidence or alter their results.

A safe queue review is a decision sequence, not a bulk command

For each request, a fictional service owner or delegated authorized reviewer should apply the following sequence:

1. Freeze intake and scope. Preserve the original queue entries, including duplicate receipts, without editing away rejected rows. Identify the approval boundary and the relevant pause or test window.
2. Determine the logical operation. Normalize only the documented fields used to distinguish identity of effect. Do not decide that two bookings match because their room names are similar.
3. Check authority. Hold entries with missing or rejected approval. Neither a patron report nor a help-desk ticket is automatically permission to update authoritative state.
4. Check prior effect. Look for a documented completed operation with the same key and exact request signature. A matching receipt may be a retry; a reused key with different content needs investigation.
5. Check object and version. Compare the specific booking object and expected version with the authoritative state in an approved copy or environment. Missing or changed objects are explicit hold conditions.
6. Apply only under controlled conditions. In a real system this would require an atomic operation and evidence of its outcome; the classroom performs no service modification.
7. Verify and notify separately. Confirm the effect against authoritative state, preserve the audit link, and communicate an accepted or rejected result only after the outcome is verified and approved.
8. Reconcile the remainder. An unresolved entry does not vanish when the queue closes. Assign an owner, next review and a supported notification state.

This is a processing model, not an instruction to execute a queue against a real library. If no suitable authoritative state or approval is available, the defensible disposition is hold. A student should never guess a current value in order to make the simulation pass.

Worked contrast: a confident but unsafe restart

Incorrect reasoning:

"R21 permits a fifteen-minute pause. Branch P later reports two manual requests. At the end of the fifteen minutes the pause must have finished, so replay both requests and tell the patrons their reservations were confirmed."

The sentence contains several unsupported conversions. R21 records tolerance, not actual execution. Branch P is released only later and reports two pending requests, not two approved or successfully processed bookings. The planned end of a pause does not prove that writes resumed or that the application, booking state, audit and manual queue passed their release tests. No canonical card supplies the two request IDs, contents or conflict status.

Defensible conditional reasoning:

"Only after the separate Branch P release may we report that the fictional service owner described a fifteen-minute pause beginning at 09:45 with two pending manual requests. Before any proposed resumption, the service owner must obtain a bounded queue manifest, determine whether each request is still authorized and consistent with current booking state, reconcile duplicate or conflicting instructions, and record both the effect and pending exceptions. The planned 10:00 end is not proof of automatic resumption. Pending items and unresolved audit or data gates remain on the handoff."

This version identifies precisely which missing evidence a later decision would need. It does not borrow the W1 test's single queued item or the P1/C1 booking comparison as if they described Branch P's two actual pending requests.

Separate the reconciliation result from return-to-service acceptance

A manual queue with every item assigned a disposition is better controlled than an unexplained queue. Still, its completion does not alone justify unrestricted booking writes.

The service owner must consider at least four separate gates: (a) the control is in the intended state after the approved change; (b) the authoritative booking values are correct under an owner-approved target; (c) relevant authorized and unauthorized tests behave as expected; and (d) changes produce adequate audit identity and time fields. The Module 3 V1 fixture separately shows why two narrow access successes do not repair an audit record with a blank actor field.

Likewise, clearing a queue by rejecting all items is not evidence of useful service. A legitimate booking function must work when permitted. A successful replay also cannot repair an unresolved account-cause hypothesis or prove that other staff-write routes are closed.

A good handoff for a manual queue reports counts with their units, such as receipts reviewed, distinct operations recognized, simulated applications, duplicate receipts, unresolved approvals and conflicts. It never reports "all patrons served" without a source supporting that much broader claim. When one logical operation has two receipts, count both receipts in the custody ledger but count its accepted business effect only once.

Failure modes and defensive review

The most important technical habit is to leave ambiguous items visible and owned. A queue is not successfully reconciled when errors have been erased; it is reconciled when each original entry has a traceable, defensible result, including exceptions.

3.13 MQ3 offline laboratory — Replaying a hypothetical queue without a live write

Objective: Distinguish duplicate receipts, conflicting key reuse, missing approval, stale state and changed processing order. This is a wholly independent synthetic laboratory. Identifiers MQ-01–MQ-08, X10–X12 and KEY-* are not canonical Harbor Library records or the two unnamed Branch P requests.

Environment: Python 3 standard library; entirely in memory. No files, network, accounts, real booking operations or external connections. The code changes only a deep-copied dictionary and does not approve any real action.

Inputs and assumptions: Each tuple contains receipt ID, operation key, booking ID, expected state version, desired room and a fictional approved flag. The baseline state is an invented exercise dictionary. Operations are processed in the printed order and one version increment represents a simulated successful change. An operation key is remembered only after a simulated apply; an unresolved hold requires independent review. These rules are intentionally narrow and do not implement durable idempotency, concurrent locking, reliable messaging or an audit service.

Before running, predict: Which receipt repeats an already simulated operation, which reuses the same key for different contents, which lacks approval, which refers to an unknown booking and which becomes stale because of an earlier simulated application?

from copy import deepcopy

BASE = {
    "X10": {"room": "Room A", "version": 2},
    "X11": {"room": "Room B", "version": 5},
    "X12": {"room": "Room C", "version": 1},
}

REQUESTS = [
    ("MQ-01", "KEY-A", "X10", 2, "Room D", True),
    ("MQ-02", "KEY-A", "X10", 2, "Room D", True),
    ("MQ-03", "KEY-B", "X11", 4, "Room E", True),
    ("MQ-04", "KEY-C", "X12", 1, "Room F", False),
    ("MQ-05", "KEY-D", "X99", 0, "Room A", True),
    ("MQ-06", "KEY-A", "X10", 2, "Room Z", True),
    ("MQ-07", "KEY-E", "X11", 5, "Room E", True),
    ("MQ-08", "KEY-F", "X10", 2, "Room B", True),
]

The next block checks approval, prior effect, object existence and expected version in that order. Run all three blocks in one Python session.

def simulate(requests):
    state = deepcopy(BASE)
    applied = {}
    results = []

    for receipt, key, booking, expected, desired, approved in requests:
        signature = (booking, expected, desired)

        if not approved:
            status = "HOLD_UNAPPROVED"
        elif key in applied:
            status = (
                "DUPLICATE_RETRY"
                if applied[key] == signature
                else "CONFLICT_KEY_REUSE"
            )
        elif booking not in state:
            status = "HOLD_UNKNOWN_BOOKING"
        elif state[booking]["version"] != expected:
            status = "HOLD_STALE_VERSION"
        else:
            state[booking]["room"] = desired
            state[booking]["version"] += 1
            applied[key] = signature
            status = "SIMULATED_APPLY"

        results.append((receipt, status))

    return results, state

The final block prints each disposition and then runs two independent order variants from the same starting state.

results, state = simulate(REQUESTS)
for receipt, status in results:
    print(f"{receipt}: {status}")

print(
    "STATE:",
    ",".join(
        f"{key}={value['room']}@v{value['version']}"
        for key, value in sorted(state.items())
    ),
)

# Negative/sensitivity control:
# Change only the order of two competing X10 requests.
for label, sequence in [
    ("ORDER_A_THEN_F", [REQUESTS[0], REQUESTS[7]]),
    ("ORDER_F_THEN_A", [REQUESTS[7], REQUESTS[0]]),
]:
    outcome, changed = simulate(sequence)
    print(
        label + ":",
        ",".join(f"{receipt}={status}" for receipt, status in outcome),
        f"X10={changed['X10']['room']}@v{changed['X10']['version']}",
    )

Compare your predictions with the MQ3 worked transcript and review in the final answer appendix.

MQ3 intake preflight: receipt identity before simulated effect

The original simulate() remembers an operation key only after SIMULATED_APPLY; it does not check whether a receipt ID was used before. In adversarial runs, two different operations both labeled MQ-01 can both return SIMULATED_APPLY. If a stale X11 request first uses KEY-Z, a different X12 request can later reuse KEY-Z and apply, because the first request was never stored in applied. These results are limits of the teaching simulator, not evidence of real queue behavior.

Review intake identity separately from any later approval, version or effect decision. The following standalone preflight retains the first signature for each receipt and each operation key; a duplicate receipt cannot seed a new key. Every original row remains in the returned output, including holds. A same-key, same-signature row is only a possible retry with unknown prior effect until a trusted outcome record is checked. Two different keys competing for the same booking may still both look new and require an owner-defined ordering rule.

REQUESTS = [
    ("MQ-01", "KEY-A", "X10", 2, "Room D", True),
    ("MQ-02", "KEY-A", "X10", 2, "Room D", True),
    ("MQ-03", "KEY-B", "X11", 4, "Room E", True),
    ("MQ-04", "KEY-C", "X12", 1, "Room F", False),
    ("MQ-05", "KEY-D", "X99", 0, "Room A", True),
    ("MQ-06", "KEY-A", "X10", 2, "Room Z", True),
    ("MQ-07", "KEY-E", "X11", 5, "Room E", True),
    ("MQ-08", "KEY-F", "X10", 2, "Room B", True),
]

def intake_preflight(requests):
    receipts, intents = {}, {}
    output = []
    for receipt, key, booking, expected, desired, approved in requests:
        signature = (booking, expected, desired)
        if receipt in receipts:
            status = "HOLD_DUPLICATE_RECEIPT_ID"
        elif key in intents and intents[key] != signature:
            status = "HOLD_CONFLICTING_OPERATION_KEY"
        elif key in intents:
            status = "POSSIBLE_RETRY_EFFECT_UNKNOWN"
        else:
            status = "NEW_INTENT_EFFECT_UNKNOWN"
        if receipt not in receipts:
            receipts[receipt] = (key, signature)
            intents.setdefault(key, signature)
        output.append((receipt, status))
    return output

for receipt, status in intake_preflight(REQUESTS):
    print(f"{receipt}: {status}")

VARIANTS = [
    ("DUPLICATE_RECEIPT_ID", [
        REQUESTS[0],
        ("MQ-01", "KEY-Z", "X11", 5, "Room F", True),
    ]),
    ("STALE_THEN_DIFFERENT_SAME_KEY", [
        ("T1", "KEY-Z", "X11", 4, "Room E", True),
        ("T2", "KEY-Z", "X12", 1, "Room F", True),
    ]),
    ("TWO_COMPETING_REQUESTS", [REQUESTS[0], REQUESTS[7]]),
]
for label, rows in VARIANTS:
    print(label + ":")
    for receipt, status in intake_preflight(rows):
        print(f"  {receipt}: {status}")

Predict the eight baseline receipts and three two-row variants in the supplemental worked transcript. Explain why MQ-02 is not yet a verified duplicate effect, why a held/stale first attempt still reserves its key's intent for conflict review, and why two competing new intents cannot be safely ordered by whichever file is read first. This in-memory check cannot authenticate approvals, guarantee atomic writes, resolve Branch P's two unnamed requests or certify durable exactly-once processing. In a real system, conditional state updates and key-to-result records would require a controlled, durable consistency design.

3.13a Safe return to service is a conjunction, not a single green result

Learning objective: Explain why a successful configuration change, a quiet audit query, correct booking values, accessible user functions and reliable attribution are different acceptance dimensions. A beginner does not need to implement transaction isolation or perform a live restore, but should be able to state what evidence would prevent premature reopening.

A session-revocation control may have the intended narrow protective effect without making booking state correct. A staff-write pause may block one known route while a scheduled job or alternate API still changes the same booking. An isolated backup comparison may match the owner-approved P1 target on B17/B18 but fail B19/B20. An authorized test may succeed while the audit record has no actor. These are not interchangeable observations, and an owner's acceptance should state its tested service scope and remaining compensating controls.

Define protection acceptance as evidence that the harmful or prohibited operation is constrained within the specified tested routes and interval. Define business acceptance as evidence that a permitted booking function remains usable under declared workflow conditions. Define data correctness against an independently approved target, not against an old backup merely because that copy has a matching digest. Define audit adequacy as the attribution and time fields required for the particular authorized test. A queue-reconciliation gate, where a pause generated manual requests, requires the service owner to know what is pending, approved, duplicated, conflicting or held. Finally, owner acceptance is a separate human decision about limited or unrestricted mode; it does not follow automatically from a test script.

Contrastive worked case A (Branch S after proper release): At 09:45 the identity administrator reports that the current desk-a session was revoked, with a narrow no-later-edit result in its queried range. That supports a bounded session-action report and a limited query observation, not a claim that every write path or all bookings are safe. Later P1/C1 reveals B19/B20 mismatches; later V1 tests one permitted and one denied operation but has a blank actor. Even after these releases, unrestricted service acceptance fails the stated data and audit checks. A responsible lead can propose monitored restricted operation only after a separate service-owner decision and a documented next review.

Contrastive worked case B (Branch P, not Branch S): The later branch card reports two unnamed pending manual requests, a fifteen-minute pause starting at 09:45, and no writes within a bounded audit interval. Those requests are not bookings already applied. Their receipt IDs, expected versions, authorizations and latest authoritative states are unavailable; therefore a student cannot decide which to replay. A queue owner must obtain a restricted manifest, detect a reused operation key or stale state, assign a supported disposition to each request, and retain unresolvable ones for owner review. The separate MQ3 fixture demonstrates these hazards with invented X-bookings, not the two Branch P requests. Reaching a nominal 10:00 expiry does not prove writes automatically resumed or that a queue item was accepted.

A deliberately important comparison: a verified duplicate receipt need not create a second business effect, whereas a different operation using the same key demands a conflict hold. Even if all pending items have dispositions, the service still needs independent booking, access and audit gates. The NIST SP 800-184 recovery guide emphasizes planning, testing and improvement, not a one-signal recovery declaration. The following classroom condition checker is a teaching model rather than NIST-mandated acceptance logic.

R3X offline lab — mandatory gates and a queue that remains pending

This standard-library Python 3 fixture is wholly invented. Each row is an independent reset; True, False and None mean declared pass, declared failure and untested, not authenticated receipts. No file, service, network or real booking is touched.

def accept(protection, service, data, audit, queue, owner):
    if protection is False: return "PROTECTION_FAIL"
    if service is False: return "SERVICE_FAIL"
    if data is False: return "DATA_FAIL"
    if audit is False: return "AUDIT_FAIL"
    if queue is False: return "QUEUE_UNRECONCILED"
    if None in (protection, service, data, audit, queue):
        return "MANDATORY_GATE_NOT_TESTED"
    if owner is not True: return "OWNER_ACCEPTANCE_PENDING"
    return "SYNTHETIC_SCOPED_GATES_PASS_ONLY"

rows = [
    ("NO_TEST", (None, None, None, None, None, None)),
    ("B19_WRONG", (True, True, False, True, True, True)),
    ("MISSING_ACTOR", (True, True, True, False, True, True)),
    ("QUEUE_PENDING", (True, True, True, True, False, True)),
    ("NO_OWNER", (True, True, True, True, True, None)),
    ("TOY_SCOPED_PASS", (True, True, True, True, True, True)),
]
for name, flags in rows:
    print(name, accept(*flags))

Exact tested output:

NO_TEST MANDATORY_GATE_NOT_TESTED
B19_WRONG DATA_FAIL
MISSING_ACTOR AUDIT_FAIL
QUEUE_PENDING QUEUE_UNRECONCILED
NO_OWNER OWNER_ACCEPTANCE_PENDING
TOY_SCOPED_PASS SYNTHETIC_SCOPED_GATES_PASS_ONLY

Adversarial interpretation: Even when the other Boolean inputs are declared pass, B19_WRONG and MISSING_ACTOR are mandatory failures. QUEUE_PENDING refuses an automatic restart merely because the technical block is over. TOY_SCOPED_PASS only shows that the declared toy fields are internally favorable; there is no approved copy, validated source clock, verified actor, independent production route test or actual owner decision. An unobserved path cannot be converted to a pass.

Reproducible task: Write a five-column plan listing the required owner, source, expected observation, failure action and status of each gate. Then reproduce all six lines, identify a failed data gate and a failed audit gate after the later P1/C1 and V1 releases, and state what the queue owner would need before reading either pending Branch P request. A full result refuses to borrow W1, MQ3 or RTE3 synthetic outcomes as Harbor Library operations and keeps incident root cause and identity questions open.

Synthesis: A temporary successful control, an accepted limited operating mode, and an eradicated cause are different decisions. Module 4 tests whether the learner can maintain these boundaries while cards are gradually released and the receiving shift lacks the author's explanations.

Worked checkpoint

If a manual request has no unambiguous disposition, reopening writes can duplicate or lose a reservation. Preserve the receipt, owner and reconciliation state; keep the service restricted until the unresolved queue and the other mandatory gates are reviewed.

Practice — Lesson 15

EX037 — Residual risk

Objective: distinguish page availability from safe, correct service. Inputs: R20, R22–R24 and a hypothetical observation that the page loads after an approved revocation; this is not the Module 4 branch result. Output: a claim/limit note with two unresolved questions, a test and owner for each, and a proposed case status. A score of 2 asks about booking correctness and adequate post-action audit coverage, and does not use R24 to claim that no personal-data access occurred.

Worked answer: EX037 in the final appendix.

EX038 — Alternative defensible choice

Objective: defend a different containment path without inventing a unique right answer. Inputs: R18–R21 and the section 3.3 matrix. Output: a pause-first proposal with two assumptions, approving role, manual-queue plan, one immediate effectiveness check, a 15-minute review time and a condition for changing to narrower session action or a longer restriction. A score of 2 states that R21 records tolerance rather than execution, and includes reconciliation and service-owner sign-off before resuming writes.

Additional queue template for EX038: At 09:37, include an empty proposed ledger with receipt ID, booking/action, claimed time, approval, expected version, prior-effect check, disposition and owner. Name at least two hold conditions and a controlled owner review before writes resume. The Branch P outcome is unavailable at this checkpoint.

RB3 extension: Additional requirement: Retain the original pause-first proposal, its two assumptions, service-owner approval requirement, queue plan, immediate effectiveness check, 15-minute review and conditional alternative. Keep the queue-ledger template empty at 09:37. Add a reversal-readiness clause: if the pause blocks essential authorized work or fails its protective purpose in a future separately verified test, what exact owner decision and bounded post-reversal checks would be required before changing that control? Do not use Branch P's later two pending requests or MQ3's synthetic rows as an observed 09:37 queue.

RB3 2-point refinement: R21 remains a tolerability statement; the proposed pause and any later reversal have independent authorization and verification. The ledger format stays empty and unchanged, and resumption remains conditional on booking/queue review and owner sign-off.

Worked answer: EX038 in the final appendix.

Continue studying — lesson 15 of at least 20

← Lesson 14 · Course lesson index · Lesson 16 →

Continue to the next authored daily lesson.

Continue studying — chapter 03 of 04

← Chapter 02 · Course lesson index · Chapter 04 →

The next chapter contains the following instructional week.