◇ OpenCyber Academy
Bachelor's · Chapter 38 · English

Chapter 38 — Capstone implementation

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 38 — Capstone implementation

Lesson map

Lesson 38 published. Map, example, EX112.

flowchart TD
  N0["Learning outcomes"]
  N1["38.1 Increments"]
  N2["38.2 Integrating controls"]
  N3["38.3 Evaluation harness"]
  N4["38.4 Defects and changes"]
  N5["38.5 Evidence"]
  N6["38.5b Code review and secure defaults"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX112"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["38.1 Increments"]
  N2["38.2 Integrating controls"]
  N3["38.3 Evaluation harness"]
  N4["38.4 Defects and changes"]
  N5["38.5 Evidence"]
  N6["38.5b Code review and secure defaults"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX112"]

[Learning outcomes]
    |
    v
[38.1 Increments]
    |
    v
[38.2 Integrating controls]
    |
    v
[38.3 Evaluation harness]
    |
    v
[38.4 Defects and changes]
    |
    v
[38.5 Evidence]
    |
    v
[38.5b Code review and secure defaults]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX112]

Learning outcomes

You will (1) implement the capstone system in increments; (2) integrate security controls and an evaluation harness; (3) manage defects and scope changes; and (4) keep evidence current for evaluation.

38.1 Increments

Build a thin end-to-end slice first (chat → model → policy → tool in sandbox), then add features. Each increment ends with tests, a short demonstration including an adversarial case, and updated documentation.

Rina's first slice let a synthetic patient ask for one appointment and booked it in the sandbox, with the policy check in place from the start. Its demonstration included one adversarial case, a document asking to book for another patient, which the check refused. Later increments added confirmation, grounding and monitoring, each with its own tests and demonstration.

38.2 Integrating controls

Implement the policy service as a separate module with its own tests. Add delegated tokens, confirmation, grounding checks and monitoring. Keep configuration in files under version control; never hard-code secrets.

Keeping the policy service separate meant its tests ran in seconds without the model, so Rina could test every rule on each change. The model provider's key was read from an environment variable set by the staging system and never appeared in the repository. Configuration such as clinic lists and booking limits sat in versioned files, so a change could be traced to a commit.

38.3 Evaluation harness

Automate the evaluation: load test suites, run conversations, collect tool logs, grade outcomes, compute rates with intervals and write a dated report. Run it nightly.

Rina's harness ran 1,000 conversations each night against staging, graded bookings from the sandbox log, and wrote a report with Wilson intervals. When a morning report showed false clinic statements rising, she traced it to a change in the structured clinic file made the day before, which had a wrong opening time; nightly runs caught it within a day.

38.4 Defects and changes

Track defects with severity. Security defects block the next milestone until fixed or formally accepted. Scope changes go through the supervisor's change log.

The confirmation screen issue described below was logged as a high-severity security defect and blocked the milestone until it was fixed. A minor wording problem in refusal messages was logged as low severity and left for a later increment. When Rina dropped the voice channel, the change went through the supervisor's change log rather than being made quietly.

38.5 Evidence

Keep evidence organised by requirement: tests, results, reports, versions. The final evaluation should require no searching.

Rina's evidence folder had one directory per requirement, each containing the test definition, the latest nightly report, the raw sandbox logs and the version file. At the final evaluation, the examiner could open the cross-patient directory and see every attempt, its outcome and the configuration tested, without asking her where anything was. Each directory also held a short text file stating what the evidence showed and its limits, written when the run was made rather than at the end.

38.5b Code review and secure defaults

Have every change touching the policy service, tokens or tool adapters reviewed by a second person. Use secure defaults: tools are denied unless explicitly allowed; logs exclude secrets and redact personal data; new prompts and model versions are not deployed until the evaluation harness passes. Write down each default so evaluators can check it.

A reviewer caught a change that logged full patient messages, which the redaction default should have prevented. The change was rejected, and a test was added that fails if any unredacted message text reaches the log, so the default is now enforced by the pipeline rather than by memory.

38.6 Worked example (synthetic LCH case)

During implementation Rina found that the confirmation screen showed the appointment summary produced by the model, which could be manipulated to look harmless. She changed the screen to render booking details directly from the structured proposal (clinic, date, time, patient) rather than model text. Nightly evaluation then showed no misleading confirmations in 300 adaptive cases.

The fix applies the same principle as the policy service: what the patient approves must come from structured data, not model text, which an injected document can shape.

38.7 Errors and limits

Building every feature first and testing at the end would have hidden the confirmation screen flaw until the final evaluation, too late to fix properly. A confirmation that shows model-written text can be made to look harmless, which defeats its purpose. Evidence gathered only at the end tends to be incomplete, since earlier runs and their versions are lost or overwritten by later runs.

Exercises

Synthesis

Implementation builds incrementally with controls and evaluation from the start; small design details, such as what a confirmation screen shows, can decide whether controls really work.

Continue studying — chapter 38 of 40

← Chapter 37 (draft) · Course chapter index · Chapter 39 (draft) →