◇ OpenCyber Academy
Technologist · Chapter 30 · English

Chapter 30 — Capstone assessment

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 30 — Capstone assessment

Lesson map

Lesson 30 published. Map, example, EX088.

flowchart TD
  N0["Learning outcomes"]
  N1["30.1 Against the brief"]
  N2["30.2 Evidence"]
  N3["30.3 Presentation"]
  N4["30.4 Reflection"]
  N5["30.6 Transition plan"]
  N6["30.7 Assessment criteria"]
  N7["30.8 Continuing"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX088"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["30.1 Against the brief"]
  N2["30.2 Evidence"]
  N3["30.3 Presentation"]
  N4["30.4 Reflection"]
  N5["30.6 Transition plan"]
  N6["30.7 Assessment criteria"]
  N7["30.8 Continuing"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX088"]

[Learning outcomes]
    |
    v
[30.1 Against the brief]
    |
    v
[30.2 Evidence]
    |
    v
[30.3 Presentation]
    |
    v
[30.4 Reflection]
    |
    v
[30.6 Transition plan]
    |
    v
[30.7 Assessment criteria]
    |
    v
[30.8 Continuing]
    |
    v
[Worked example] --> [Exercise EX088]

Learning outcomes

You will (1) evaluate the capstone against its brief; (2) present results to assessors and a client panel; (3) reflect on limits and next steps; (4) document a transition plan to real use.

30.1 Against the brief

Each requirement is checked: imports work, labels are preserved, matching is exact, expiries are flagged, export filter refuses restricted items, tests pass in one command.

Each requirement in the brief maps to a test or a demonstration. Exact matching is shown by the near miss pay-portal.example.org not matching, and the export filter by an AMBER item being refused. Tests passing in one command shows the whole pipeline can be checked by an assessor in minutes. Each check's result is shown in a single table, so assessors can see at a glance which requirements are met.

30.2 Evidence

Assessors see test outputs, the build log, code review comments and a short demonstration on synthetic data.

Each kind of evidence covers something the others do not: test outputs show correctness, the build log shows reasoning and problem-solving, review comments show how feedback was handled, and the demonstration shows that the pipeline runs. Together they let assessors judge the work without relying on Lena's own description. The demonstration runs the same tests assessors can run themselves, so nothing shown depends on Lena's own machine. The build log and review comments are submitted with the code, so assessors can follow how the final version came about.

30.3 Presentation

Ten minutes: purpose, design diagram, demonstration, results, limits, next steps. A panel member from MCX asks about handling; Lena explains the fail-closed export.

A fail-closed export refuses any item whose label is not CLEAR or GREEN, including unknown or misspelt labels. The question from an MCX panel member reflects the exchange's main concern, that members' restricted items never leave CMS by mistake. Lena's answer shows that the filter's design follows directly from TLP handling rules. The demonstration includes the refused AMBER item, so the panel sees the filter work as well as hears it explained.

30.4 Reflection

What went well (modular design, tests); what was hard (format changes); what she would do differently (write the end-to-end test first).

Format changes were hard because the matcher fault in the worked example came from a new log column, the kind of change real feeds make often. Writing the end-to-end test first would have shown expected outputs from the start, so each module could be built against them. Her reflection names specific events from the build rather than general impressions. Each point in the reflection is linked to a dated entry in the build log, so assessors can check the account.

30.5 Limits

Synthetic data is cleaner than reality; the pipeline has not met real feeds; performance at scale is untested.

Each limit points to a step in the transition plan. Validation on real past data addresses the gap between clean synthetic inputs and messy feeds, and a report-only run shows behaviour on real data without risk. Performance at scale would be measured during that run, against the ten-minute runtime used in chapter 27. Lena states these limits in the presentation as well as the report, so the panel does not overestimate readiness.

30.6 Transition plan

To use it at CMS: validation on real past data, report-only run, change approval, service account, monitoring, owner and documentation — as in chapter 27.

Following chapter 27 means the pipeline is treated as production software rather than a student project. It must pass validation, run in report-only mode, go through change approval and run under a service account with minimum rights. The client panel's two requested changes are added to the transition requirements. The plan names CMS's supervisor as the likely owner, pending the manager's agreement, so responsibility is clear from the start.

30.7 Assessment criteria

Correctness, safety, code quality, documentation, reflection and presentation, each scored with the published rubric.

Publishing the rubric in advance lets Lena check her work against the same criteria the assessors use. Safety is scored separately from correctness, because a pipeline that works but could share restricted data would not be acceptable. Reflection and presentation are scored too, because explaining work is part of the job. Each criterion has rubric levels with descriptions, so two assessors marking the same work should reach similar scores. The scores and written comments return to Lena with the result, so she can see which criteria held her mark down.

30.8 Continuing

Graduates move toward analyst roles; the next courses build analysis and leadership skills on these foundations.

The habits built here, careful handling, exact matching, honest limits and clear notes, are the ones Lena will rely on as an analyst. Later courses add deeper analysis, assessment writing and team leadership. Her placement contacts and capstone give her concrete examples to draw on when she applies for analyst roles. Lena's capstone report and placement journal stay in her portfolio, giving evidence of these habits to future employers. The pipeline itself may continue at CMS if the transition plan is approved, which would give her a working tool to point to.

Feedback from the client panel

The panel asks for two changes before any real use: weekly notes in the council's house style, and an extra column showing which resident-facing service each match affects. Lena records both as transition requirements rather than rushing them into the assessed version, keeping the submitted work stable while the client's needs are captured for the next phase.

Worked example

A panel member asks whether the pipeline could share AMBER items with partner councils automatically. Lena answers no: AMBER restricts sharing to CMS and its clients, and outward sharing needs human approval. The panel notes the answer as evidence of handling discipline.

Synthetic case. The panel is invented.

Errors and limits

Assessment captures one moment; skills grow with practice.

Exercises

Synthesis

Assess against the brief with evidence, present clearly, reflect honestly and plan a careful transition.

Continue studying — chapter 30 of 30

← Chapter 29 (draft) · Course chapter index · Worked results ↓ · Next course via chapter index →