◇ OpenCyber Academy
Engineering · Chapter 46 · English

Chapter 46 — Engineering capstone II

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 46 — Engineering capstone II

Lesson map

Lesson 46 published. Map, example, EX136.

flowchart TD
  N0["Learning outcomes"]
  N1["46.1 Building"]
  N2["46.2 Testing"]
  N3["46.3 Timing"]
  N4["46.4 Unexpected problem"]
  N5["46.5 Documentation"]
  N6["46.6 Handover readiness"]
  N7["Code quality"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX136"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["46.1 Building"]
  N2["46.2 Testing"]
  N3["46.3 Timing"]
  N4["46.4 Unexpected problem"]
  N5["46.5 Documentation"]
  N6["46.6 Handover readiness"]
  N7["Code quality"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX136"]

[Learning outcomes]
    |
    v
[46.1 Building]
    |
    v
[46.2 Testing]
    |
    v
[46.3 Timing]
    |
    v
[46.4 Unexpected problem]
    |
    v
[46.5 Documentation]
    |
    v
[46.6 Handover readiness]
    |
    v
[Code quality]
    |
    v
[Worked example] --> [Exercise EX136]

Learning outcomes

You will (1) build and test the capstone system; (2) measure against success criteria; (3) handle unexpected problems; (4) document the implementation.

46.1 Building

Kwame builds the verifier in a memory-safe language using standard libraries where possible, with its own CI, signing and provenance, so it meets the same rules it enforces.

A verifier that skipped the controls it checks would be an obvious weak point: an attacker who altered the verifier could make every mismatch look clean. Building it through the same pipeline means its own releases are signed, have provenance and pass the dependency gate. Using standard libraries keeps its dependency graph small, which reduces the chance that a third-party package undermines it.

46.2 Testing

The test set of 20 synthetic releases includes 5 mismatches: altered file, missing signature, wrong builder, unreviewed commit and an extra file. All 5 are detected; no false alerts on the 15 good releases.

Each mismatch type tests a different check: digest comparison, signature verification, builder policy, review status and file listing. Detecting all five shows every check works, and the absence of false alerts on good releases shows the checks are not over-sensitive. The test set is stored with the code and run on every change.

46.3 Timing

Each release is checked in a median of 7 minutes after detection; the slowest takes 22 minutes, within the one-hour target.

Reporting the median and the slowest case together shows both typical behaviour and the margin against the target. Most of the time is spent waiting for store metadata rather than computing. A slowest case of twenty-two minutes leaves plenty of room under the hour, even if store responses slow down. Kwame records the timing of every check, so a gradual slowdown is visible long before it threatens the target.

46.4 Unexpected problem

App store metadata sometimes lags; the verifier now retries for up to 45 minutes before alerting "not yet available" rather than "mismatch".

Reporting a mismatch when the data simply has not arrived would produce false alerts, which the success measures limit to fewer than one a month. The new status separates two situations that need different responses. If metadata is still missing after the retry window, the alert says so plainly, and on-call checks the store rather than starting a tampering investigation. Kwame adds this case to the test set, using a synthetic release whose metadata appears late.

46.5 Documentation

Operations guide, architecture notes, runbook for alerts and a test report.

Each document serves a different reader. The operations guide explains how to run and update the service. The architecture notes record design choices and their reasons. The runbook tells on-call engineers what each alert means and what to do first. The test report gives evidence that the success measures were met. All four are kept in the repository and reviewed with code changes, so they stay accurate after Kwame leaves.

46.6 Handover readiness

Coastline's platform team reviews the code and runs it in staging for two weeks.

Running in staging lets the platform team see the verifier under realistic conditions before relying on it. During the two weeks, its engineers handle any alerts themselves, with Kwame available to explain. Their code review checks that they can understand and change the service without him. Any issues found are fixed before the service moves to production, and the team signs off only when it is confident it can operate the verifier alone. That sign-off is the readiness test.

Code quality

The verifier's code is reviewed by two Coastline engineers. Its own tests cover each policy rule and the retry logic. Static analysis runs on every change, and dependencies are kept minimal: only the standard library and one well-maintained library for signature checking. Kwame applies every rule from the course to his own project, so it can be trusted as a security tool.

Monitoring the verifier

The verifier reports its own health: last successful check, queue length and error counts. If it stops checking for more than two hours, an alert fires, so a silent failure of the checker cannot go unnoticed.

Worked example

During staging, a real release triggers an alert: an extra configuration file. Investigation shows the vendor's build tool added it in a new version. It is harmless, but the release process is updated to include it in the signed build so future checks pass.

Synthetic case. Results are synthetic.

Errors and limits

A test set represents known mismatch types; unknown types may evade it.

Exercises

Synthesis

Build to the same standard you enforce and measure against agreed criteria.

Continue studying — chapter 46 of 50

← Chapter 45 (draft) · Course chapter index · Chapter 47 (draft) →