◇ OpenCyber Academy
Engineering · Chapter 50 · English

Chapter 50 — Final defense of design and measured results

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 50 — Final defense of design and measured results

Lesson map

Lesson 50 published. Map, example, EX099.

flowchart TD
  N0["Learning outcomes and connection"]
  N1["50.1 What the defense tests"]
  N2["50.2 Preparing"]
  N3["50.3 Answering counterexamples"]
  N4["50.5 Revisions"]
  N5["Chapter synthesis"]
  N6["Chapter 1"]
  N7["Chapter 2"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX099"]

Logical drawing

flowchart LR
  N0["Learning outcomes and connection"]
  N1["50.1 What the defense tests"]
  N2["50.2 Preparing"]
  N3["50.3 Answering counterexamples"]
  N4["50.5 Revisions"]
  N5["Chapter synthesis"]
  N6["Chapter 1"]
  N7["Chapter 2"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX099"]

[Learning outcomes and connection]
    |
    v
[50.1 What the defense tests]
    |
    v
[50.2 Preparing]
    |
    v
[50.3 Answering counterexamples]
    |
    v
[50.5 Revisions]
    |
    v
[Chapter synthesis]
    |
    v
[Chapter 1]
    |
    v
[Chapter 2]
    |
    v
[Worked example] --> [Exercise EX099]

Learning outcomes and connection

After this chapter you can (1) present a response system's design rationale and evidence to a critical panel; (2) maintain a safety and security argument under challenge; (3) answer counterexamples by reference to evidence, alternatives and limits; and (4) record required revisions and close them. It completes the book by integrating Chapters 44–49.

50.1 What the defense tests

The panel tests whether the engineer understands the system well enough to explain why it is built as it is, what evidence supports each claim, what alternatives were rejected and why, and where the system will fail. It is not a sales pitch. A strong defense makes its limits visible before the panel finds them. In the exchange below, the candidate makes limit L6 visible before the panel finds it. The panel will probe most where the dossier lists exceptions, open defects and not-verified items.

50.2 Preparing

50.3 Answering counterexamples

A counterexample is a scenario in which a claim appears to fail. Four honest responses exist: (1) refute with evidence that the scenario was tested and the claim held; (2) qualify — the claim holds under stated conditions that exclude the scenario; (3) concede — the claim fails in that scenario, recorded as a limit or defect; (4) investigate — unknown, with a plan to find out. Bluffing is the only wrong answer. The panel weighs how well the engineer distinguishes these, more than how many questions are refuted.

50.4 Worked example (synthetic): defense exchange

Panel: "Your controller holds isolations during a policy-service partition. Suppose an attacker causes the partition deliberately after triggering a false-positive isolation of a critical server. You have now locked in disruption."

Candidate: "Servers are not automatically isolated — server isolation is gated (DD-3, Chapter 17); so the false positive would need an approver. If an approver was deceived, the isolation would persist under its lease during the partition. The lease extension during partition is limited to one 60-minute extension with an alert (Chapter 33); the duty manager can reverse via the break-glass path. So the disruption is bounded at the original lease plus 60 minutes unless a human intervenes earlier. I concede that the design does not detect a deliberately induced partition as an attack; that is listed as limit L6, and the policy-service health is monitored independently, which would at least show the partition. A future change could treat partition during active containment as a security event."

Panel: "Your tenant-isolation claim was tested with two tenants. Real deployments have hundreds."

Candidate: "Qualify: the tests show the predicate is enforced on target tenant for every action type; the predicate does not depend on the number of tenants, which I verified by inspection and an exhaustive test over generated tenant sets up to 50. Performance at hundreds of tenants is not verified; that is listed as not-verified under R17."

50.5 Revisions

The panel's required revisions go into a register: item, source question, required change or evidence, owner, due date, verification. Closing the register — with evidence, not assertion — completes the capstone. Optional recommendations are recorded separately.

In the exchange below, a revision could ask for performance evidence at hundreds of tenants under R17. Its register entry would name the tenant-scale question as the source, a load test as the required evidence, and an owner with a due date. A revision that changes a claim must also update the dossier entry that supported it.

50.6 Errors and limits

The worked exchange shows the candidate avoiding the first three errors. The partition answer qualifies the claim with the 60-minute lease extension and concedes limit L6. The tenant answer separates what inspection and tests up to 50 tenants show from performance at hundreds, which is listed as not verified under R17. Any panel revision on R17 must close with a measured result, not an oral promise.

Exercises — chapter 50

EX099 — Counterexample responses. Objective: practise honest answers. Inputs: three panel questions: (a) "What if the anchor key service is compromised?" (b) "Your latency test used synthetic traffic — why should we trust it?" (c) "Could an approver approve their own request using a second account?" Steps: answer each, labelling the response type (refute, qualify, concede, investigate) and citing the evidence. Expected evidence: three answers. Rubric: 3 = honest, evidence-linked, correct labels; 2 = partial; 1 = bluffing; 0 = none.

EX100 — Revision register. Objective: close the loop. Inputs: the two panel exchanges in 50.4. Steps: write register entries for the required revisions they imply, with verification method. Expected evidence: entries. Rubric: 3 = specific changes (partition-as-security-event detection, tenant-scale performance test) with owners, due dates and verification; 2 = partial; 1 = vague; 0 = none.

Chapter synthesis

The defense shows that the engineer knows why the system is built as it is, what evidence supports each claim and where it fails. Answer counterexamples by refuting, qualifying, conceding or investigating — never bluffing — and close the revisions with evidence.

Continue studying — chapter 50 of 50

← Chapter 49 · Course chapter index · Next course via chapter index →

This is the final chapter of this course; the index links the next course in the full study path.

Worked answers

EX099. (a) Qualify/concede: "If the key service is compromised, an attacker who also holds write access to the store could rewrite and re-anchor the chain; the claim of tamper-evidence holds only while key custody is separate (assurance case defeater D2). Mitigations in the design: separate administrators and key-use logging to an independent store; daily anchors also sent to an external party, which would expose a rewrite of earlier days. Evidence: inspection IR-2, Lab 19-style tests." (b) Qualify: "The test shows the pipeline meets p99 ≤ 5 min for the synthetic traffic shape and 5× volume (LT-3, 3 runs); it does not show behaviour under real burst shapes or provider delays — stated as a threat to validity with a planned replay test." (c) Refute/qualify: "The workflow prevents self-approval by identity; a second account held by the same person would defeat that unless accounts map to one person. Our IdP stub enforces one person per identity (AT-02d), but in deployment this depends on the identity governance process — a stated assumption, not a property of our system."

EX100. RV-1: detect policy-service partition during active containment as a security event — change: correlation rule and alert to the duty manager; owner: candidate; due: 4 weeks; verification: fault-injection test where the partition coincides with an active isolation and the alert fires within 2 min. RV-2: tenant-scale verification — change: generated test with 500 tenants and performance measurement of the authorisation predicate and queries; owner: candidate; due: 4 weeks; verification: test report with latency percentiles, and R17 status updated in the dossier. Each closed with artefacts linked in the revision register.