◇ OpenCyber Academy
Technologist · Chapter 30 · English

Chapter 30 — Capstone assessment

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 30 — Capstone assessment

Lesson map

Lesson 30 published. Map, example, EX059.

flowchart TD
  N0["Learning outcomes"]
  N1["30.1 Criteria"]
  N2["30.2 Live demonstration"]
  N3["30.3 Questions"]
  N4["30.4 Handover"]
  N5["30.6 Practical activity"]
  N6["30.7 Self-assessment"]
  N7["30.9 After the assessment"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX059"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["30.1 Criteria"]
  N2["30.2 Live demonstration"]
  N3["30.3 Questions"]
  N4["30.4 Handover"]
  N5["30.6 Practical activity"]
  N6["30.7 Self-assessment"]
  N7["30.9 After the assessment"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX059"]

[Learning outcomes]
    |
    v
[30.1 Criteria]
    |
    v
[30.2 Live demonstration]
    |
    v
[30.3 Questions]
    |
    v
[30.4 Handover]
    |
    v
[30.6 Practical activity]
    |
    v
[30.7 Self-assessment]
    |
    v
[30.9 After the assessment]
    |
    v
[Worked example] --> [Exercise EX059]

Learning outcomes

You will (1) evaluate your capstone against criteria; (2) demonstrate it live; (3) answer technical questions; and (4) hand over artefacts for reuse.

30.1 Criteria

Assessment criteria: completeness of the slice; correctness of configuration (hybrid preferred, fallback controlled); quality of automation and monitoring; runbooks exercised; reproducibility; residual risk and exceptions documented; clarity of decisions and evaluation.

Each criterion maps to evidence in the repository: configuration correctness to the rendered profile and logs, automation to the gate and its test, monitoring to the dashboard rebuilt from recorded logs, runbooks to the tabletop record. Examiners score each from 0 to 3 using the same scale as the exercises, so a candidate can predict the outcome from the evidence.

30.2 Live demonstration

Demonstrate: build the lab; show negotiation of hybrid KEX; trigger a fallback by reverting configuration and show the alert; follow the runbook to recover; show the CI gate blocking a bad change.

Each step has a time and an observable result: the lab builds without manual steps; logs show the hybrid algorithm; a configuration revert triggers the fallback alert; the runbook restores the profile and the alert clears; a pull request with a reordered list fails the gate. Rehearse the failure steps, since they are the ones that go wrong live.

30.3 Questions

Expect: "Why is the server's order important?", "What does hybrid not protect?", "How would you migrate host keys?", "What happens if ML-KEM is weakened?", "How do you know the scanner is right?" Answer with evidence from your repository.

Good answers cite the repository. On order: the lab record showing a misordered list negotiating X25519. On what hybrid does not protect: authentication, recorded past traffic and any classical hop. On a weakened ML-KEM: the combiner keeps the session as strong as X25519. On the scanner: its validation record against the three lab servers.

30.4 Handover

Leave a README, decision records, runbooks and open issues so another team can extend the work.

The README states what the slice does, how to build it and what is out of scope. Open issues are written for someone who was not on the team, with the evidence that prompted each and a suggested first step. Decision records and runbooks stay in the repository so the next team inherits the reasoning, not just the configuration.

For the SFTP slice, the emulation-only client weakness becomes issue 1, with the matrix row and the reason no real device was available.

30.5 Worked example

Assessment of the SFTP capstone: build in 9 minutes; hybrid KEX shown in logs; fallback triggered and alert raised in 2 minutes; recovery by runbook in 4 minutes; CI gate blocked a reordered KexAlgorithms line. Examiner asks about host keys; student explains forge-later timing, SSH certificates as an agility step and waiting for post-quantum host-key algorithms in OpenSSH.

The timings show the slice is operable, not only configured: an alert within two minutes of fallback and recovery within four by runbook. The host-key answer earned full marks because it separated key exchange from authentication and named a concrete agility step rather than promising future algorithms.

30.6 Practical activity

Rehearse your demonstration with a peer acting as examiner; time each step.

Run the full demonstration end to end with a peer who asks the five questions from the assessment list without warning. Time each step and note where you hesitated or needed a manual command. Fix any step that depended on something not in the repository, then rehearse again. A demonstration that runs within its time twice in a row is ready; one that worked once is not.

In KLS rehearsals the slowest step was usually rebuilding the lab after the CI gate demo; one team cut it from nine minutes to two by caching the container images in the repository's build script.

30.7 Self-assessment

Before the assessment, score yourself against each criterion with evidence links, and note one weakness you would fix with more time. Examiners value candidates who see their own gaps; it shows the judgment that operations teams rely on. Include the self-assessment in the handover so future teams know where to start.

For the SFTP capstone, a score of 3 on verification links to the 3,000-session evaluation run, not just to the dashboard.

Name one weakness, the evidence that shows it and the fix you would make with another week. For the SFTP capstone a fair example is a client type tested only in emulation. Note the weakness openly; examiners treat it as judgment, not as a lost mark.

30.8 Common errors and limits

A demo that needs a hand-typed command not in the repository fails reproducibility on the spot. A candidate who says hybrid key exchange protects the SFTP host's identity has confused key establishment with authentication; the host key is still Ed25519. And a capstone with no README or open issues leaves the next team to rediscover everything. Each is visible to an examiner within minutes.

30.9 After the assessment

Within a week of the assessment the student files the examiner's questions and their answers with the capstone record, fixes any documentation gap the examiner found, and hands KLS operations a one-page summary of what the SFTP service now negotiates and what is still classical. That summary is the starting point for the next migration step.

It also states the next step explicitly: host certificates for SSH, then post-quantum host keys when available.

Those two steps become the first issues in the next team's repository, each with the evidence that prompted it.

Exercises

Synthesis

Assessment shows a working, monitored, recoverable slice, honest answers about its limits, and artefacts that others can reuse.

Continue studying — chapter 30 of 30

← Chapter 29 (draft) · Course chapter index · Worked results ↓ · Next course via chapter index →