Chapter 11 — Applied project
Lesson map
Lesson 11 published. Map, example, EX051.
flowchart TD
N0["Learning outcomes"]
N1["11.1 Project brief"]
N2["11.2 Non-goals"]
N3["11.3 Pre-registration"]
N4["11.4 Handover"]
N5["11.5 Ethics cover"]
N6["11.6 Time box"]
N7["11.7 Narrative"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX051"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["11.1 Project brief"]
N2["11.2 Non-goals"]
N3["11.3 Pre-registration"]
N4["11.4 Handover"]
N5["11.5 Ethics cover"]
N6["11.6 Time box"]
N7["11.7 Narrative"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX051"]
[Learning outcomes]
|
v
[11.1 Project brief]
|
v
[11.2 Non-goals]
|
v
[11.3 Pre-registration]
|
v
[11.4 Handover]
|
v
[11.5 Ethics cover]
|
v
[11.6 Time box]
|
v
[11.7 Narrative]
|
v
[Worked example] --> [Exercise EX051]
Learning outcomes
You will (1) deliver a specialist project for MCH; (2) keep non-goals; (3) evaluate with pre-registration; (4) hand over to operations.
11.1 Project brief
The applied project builds a scenario-scoring sheet and an assurance linter integrated with FindingsDesk fields, evaluated on synthetic past exercises. The scoring sheet reuses the lab_scenario formula, two times learning minus risk, and the linter reuses the four forbidden phrases. FindingsDesk field integration is part of the brief, not optional polish. The brief forbids live participant scanning, exploit modules and LLM auto-bury of findings, and the deliverable must run when Amira may not type during solo-run week, which is how the project proves it can be handed over.
11.2 Non-goals
The project has three non-goals: no live participant scanning, no exploit modules and no LLM auto-bury of findings. They are checked again at every biweekly checkpoint, where a quiz asks which non-goal you nearly violated this sprint. Amira answers first with her own near-miss about an LLM summary feature, so juniors see that candour is expected; hearing a senior admit a near miss makes it easier for juniors to raise their own. A repeated near miss on the same non-goal becomes a topic for the next checkpoint.
11.3 Pre-registration
Two measures are pre-registered: the precision of forbidden-phrase catches, and the ranking correlation with the white team's expert order on a frozen set. The frozen set is a fixed batch of synthetic past exercises that nobody may edit once the tag is pushed, and tags go in git before evaluation runs. Amira will not tune thresholds after seeing how her favourite scenarios score. Any amended tag would need a decision-log note, the same rule that governs scenario thresholds in controlled evaluation.
11.4 Handover
At handover, operators run the linter alone and Amira may not type. The setup is a freshly created operator account with empty shell history, the README and a sealed envelope of ROE excerpts; the operator must produce a linted board pack from those alone. Amira is reachable only by phone that week, and if the operator phones her, the call is logged as a documentation defect against the README, not treated as a quiet rescue that hides gaps. The defect log is the main output of handover week.
11.5 Ethics cover
Every project packet opens with a synthetic-only declaration on page one, and reviewers check that declaration before reading any code in the repository. The check is mechanical: merges reject cover sheets that lack the word "synthetic", and a rejected merge is bounced to the author with the cover sheet template attached. The same candour applies to near-misses: answers from others stay anonymous, while Amira's own near-miss stays named so candour is modelled from the top. The cover is the first thing a reviewer reads.
11.6 Time box
The project runs for eight weeks with biweekly checkpoints, which keep it inside a specialist time box. Amira refuses open-ended build time that starves catalogue work, even when the tooling looks promising. The checkpoints quiz the non-goals and show whether solo-run readiness is real before the clock ends. The week-seven solo runs in the worked example fell inside that clock, leaving one checkpoint for fixes. A project that needs a ninth week has to ask for it through a decision object, not take it by drift.
11.7 Narrative
The checkpoint quiz takes five minutes at the start of each biweekly checkpoint, before any demo of new features, so non-goals are discussed before anyone is excited about what was built. Amira answers first with her LLM summary near-miss. Ethics covers are checked for "synthetic" before project repositories accept merges. Solo-run week forbids her typing, so operators must succeed from the docs alone. Each of these habits is small, and together they keep the non-goals visible in every sprint of the project.
11.8 Near-miss log
Near-miss answers are stored anonymously when others share them, logged by sprint number only so patterns across the eight weeks stay visible. Amira's own near-miss stays named, which signals to juniors that admitting one carries no penalty; candour that only juniors show is not candour, and leaders go first. She repeats the habit in decision clinics, opening with the LLM summary feature she nearly built and then asking the room for one of their own. The log is reviewed at the final checkpoint for repeated patterns.
11.9 Solo-run week
In cohort one, the operator phoned Amira twice during solo-run week. Both calls were logged as documentation defects and became README patches the same afternoon, followed by a thank-you to the operator. Each patch quotes the README step the operator could not complete, exactly as they read it, so the fix addresses the real gap rather than Amira's memory of it. The operator account is deleted after the week, so the next cohort cannot inherit any shell history. Cohort two made only one call, about the same README section.
11.10 Sealed envelopes
The sealed ROE envelopes carry wax stamps, a deliberate piece of theatre that reminds people the ROE is special: opening one should feel different from opening routine paperwork. Juniors remember that feeling when they later handle production ROEs, so here theatre serves care, not swagger. The stamp is broken only by the operator on the morning of solo-run week, in front of a witness who logs the time, which also gives the handover a clear starting record. The broken seal and the logged time are filed with the handover notes.
Worked example (synthetic MCH)
Amira's project catches 100% of planted forbidden phrases on the frozen set; operators complete solo runs on week seven; non-goals hold.
Synthetic case — Meridian Clearing House (MCH). The clearing organisation, Amira Hassan and all figures are invented for teaching.
Errors and limits
Threat-led and clearing-system tests require specialised legal authorization. This book uses synthetic scenarios only and never supplies exploit code or live targeting advice for market infrastructures.
Exercises
- EX051 — Non-goals. Name two project non-goals. Rubric: 3 = two of no live scan, no exploits, no LLM auto-bury; 2 = one; 1 = vague; 0 = none.
- EX052 — Metric. Give one pre-registered metric. Rubric: 3 = forbidden-phrase catch precision or ranking correlation; 2 = vague; 1 = vibes; 0 = none.
- EX053 — Solo run. Why must Amira not type during handover? Rubric: 3 = docs must suffice for operators; 2 = partial; 1 = she should type; 0 = none.
- EX054 — Cover. What declaration opens the project packet? Rubric: 3 = synthetic-only; 2 = partial; 1 = live OK; 0 = none.
- EX055 — Week seven. What happens in week seven in the worked example? Rubric: 3 = operator solo runs; 2 = partial; 1 = go live on participants; 0 = none.
Synthesis
The applied project ships guarded tooling with solo-run proof and intact non-goals.
Continue studying — chapter 11 of 12
← Chapter 10 (draft) · Course chapter index · Chapter 12 (draft) →