◇ OpenCyber Academy
Engineering · Chapter 46 · English

Chapter 46 — Engineering capstone II

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 46 — Engineering capstone II

Lesson map

Lesson 46 published. Map, example, EX136.

flowchart TD
  N0["Learning outcomes"]
  N1["46.1 Results against objectives"]
  N2["46.2 Comparison with paper"]
  N3["46.3 Qualitative feedback"]
  N4["46.4 Exceptions"]
  N5["46.5 Interpretation"]
  N6["46.6 Recommendations"]
  N7["46.7 Limitations"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX136"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["46.1 Results against objectives"]
  N2["46.2 Comparison with paper"]
  N3["46.3 Qualitative feedback"]
  N4["46.4 Exceptions"]
  N5["46.5 Interpretation"]
  N6["46.6 Recommendations"]
  N7["46.7 Limitations"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX136"]

[Learning outcomes]
    |
    v
[46.1 Results against objectives]
    |
    v
[46.2 Comparison with paper]
    |
    v
[46.3 Qualitative feedback]
    |
    v
[46.4 Exceptions]
    |
    v
[46.5 Interpretation]
    |
    v
[46.6 Recommendations]
    |
    v
[46.7 Limitations]
    |
    v
[Worked example] --> [Exercise EX136]

Learning outcomes

You will (1) evaluate the pilot against objectives; (2) analyse quantitative and qualitative data; (3) recommend next steps; (4) report limitations.

46.1 Results against objectives

O1: 212 of 212 new items registered (100%). O2: 0 examinations on unverified copies; 3 verification failures caught (all network copies, re-copied successfully). O3: custody complete for all items. O4: median satisfaction 4 of 5, baseline 3 of 5.

Each result maps directly to its objective, so the reader does not need to interpret them. The three caught failures matter for O2: they show the blocking verification was exercised in real use, not just in tests. All three involved network copies, consistent with the earlier research note, and all were resolved by copying again.

46.2 Comparison with paper

Prior quarter paper records: 17 of 198 items had incomplete custody entries. EvidenceDesk: 0 of 212.

Seventeen of 198 is about 8.6 percent of items with gaps, against none in the pilot. The comparison is not perfect, since the quarters differ in cases and staff attention, but required fields and blocking checks plausibly explain most of the difference. Priya lists which fields were most often missing on paper, mainly seal numbers and transfer times. She also checks that both quarters received similar mixes of item types, so a quarter dominated by phones could not explain the gap alone.

46.3 Qualitative feedback

Examiners liked blocking verification once used to it; disliked the extra seal-number confirmation step. One said the custody export "will save hours in court preparation".

Initial dislike of blocking verification faded once examiners saw it catch real problems. The seal-number complaint is specific and fair: confirming the number required retyping it in full. Feedback is reported as given, including complaints, because a report that only quotes praise would be less credible to the lab's manager. A simpler confirmation is planned in response.

46.4 Exceptions

Five exceptions over the pilot: two during updates, three during a power cut. All reconciled within one working day.

Exceptions during updates came from staff trying to log in during the change window; the fix is a clear notice on screen. The power cut stopped the server, and paper forms bridged the gap as designed. Reconciling all five within a day shows the degradation procedure worked in practice, not only in rehearsal. Each exception record includes the paper form scan, the time of re-entry and who checked it.

46.5 Interpretation

The pilot met all objectives. Results reflect one lab, three months and motivated staff aware of the trial.

Meeting the objectives supports adoption, but Priya does not claim the same results would follow at a larger lab or over years. Motivated staff who know they are in a trial may be more careful than usual. Writing that down protects the conclusion from overreach and tells future readers what still needs checking. A follow-up after a full year would test whether the pilot's gains last.

46.6 Recommendations

Adopt EvidenceDesk permanently; schedule revalidation for the next imager update; simplify seal confirmation while keeping the field required; continue monthly metrics.

Each recommendation follows from a pilot finding: adoption from the objectives, revalidation from the validation plan's triggers, the seal-step change from feedback, and monthly metrics from the limits. Keeping the seal field required but easier to complete answers the complaint without losing the completeness gain. Each item has an owner and a date. Recommendations are listed in priority order, with adoption first.

46.7 Limitations

Hawthorne-style effects (staff behaving differently when observed), short duration, no comparison lab.

Observed staff may follow procedures more closely than usual, inflating completeness. Three months may not reveal slow problems, such as media ageing or staff drift back to old habits. Without a comparison lab running only paper over the same period, some improvement could reflect general changes at Northfield. The monthly metrics will show whether results hold once attention fades. Each limitation also suggests a follow-up that could reduce it.

46.8 Handover decision

Northfield's manager approves adoption, with the IT officer as system owner.

Naming a system owner gives one person responsibility for updates, backups and revalidation after Priya leaves. The IT officer already made a release during the maintainability handover, which made the decision easier. Priya remains available for questions for three months, and the arrangement is written into the handover record. The manager's approval is recorded with its date and the conditions attached, including the revalidation schedule, and filed with the validation report so future auditors can find it.

Cost and effort

Priya also reports cost: a reused server, about 120 hours of her time and 30 hours of staff training. Intake time per item rose by roughly two minutes, while court preparation for custody questions fell sharply according to examiners. She presents both so the manager weighs the real trade-off, and notes which figures are measured and which are estimates from staff.

Worked example

Priya's summary: "Over three months, every new item was registered and no examination began on an unverified copy; incomplete custody records fell from 17 of 198 to 0 of 212. These results come from one lab during an observed pilot."

Synthetic case. Pilot results are invented.

Errors and limits

Improvement during a pilot may fade; keep measuring after adoption.

Exercises

Synthesis

Evaluate against stated objectives, compare with baseline, hear users and state limits plainly.

Continue studying — chapter 46 of 50

← Chapter 45 (draft) · Course chapter index · Chapter 47 (draft) →