Chapter 9 — Application and software supply chain security: leadership and assurance
Lesson map
Lesson 09 published. Map, example, EX041.
flowchart TD
N0["Learning outcomes"]
N1["9.1 Leadership"]
N2["9.2 Assurance"]
N3["Lab — assurance indicators"]
N4["9.3 Lab result"]
N5["9.4 Indicators"]
N6["9.5 Independent checks"]
N7["9.6 Gaps"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX041"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["9.1 Leadership"]
N2["9.2 Assurance"]
N3["Lab — assurance indicators"]
N4["9.3 Lab result"]
N5["9.4 Indicators"]
N6["9.5 Independent checks"]
N7["9.6 Gaps"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX041"]
[Learning outcomes]
|
v
[9.1 Leadership]
|
v
[9.2 Assurance]
|
v
[Lab — assurance indicators]
|
v
[9.3 Lab result]
|
v
[9.4 Indicators]
|
v
[9.5 Independent checks]
|
v
[9.6 Gaps]
|
v
[Worked example] --> [Exercise EX041]
Learning outcomes
You will (1) lead a programme for application and supply-chain security; (2) provide assurance to boards and regulators; (3) choose assurance indicators; (4) respond to assurance gaps.
9.1 Leadership
Hannah leads a programme with three goals for the year: every app release verifiable (provenance and signatures), every SDK update reviewed, and critical dependency findings fixed within 14 days.
Each goal is tied to a measure the chapter later reports, so progress can be checked quarter by quarter. The goals came from the gaps found in chapter 2 and the February incident, which helps engineers see why they matter. Hannah presents the three goals to the CISO with the resources each needs, so the programme's cost is agreed at the start.
9.2 Assurance
Assurance means giving decision-makers justified confidence. The board and the regulator want evidence that controls work, not just that they exist.
At Northbridge, assurance for the SDK programme rests on three things: indicators reported each quarter, records that let someone check an individual release, and periodic independent checks by internal audit. Each part supports the others, so a gap in one is noticed elsewhere. The board receives the summary, and the detailed records stay available for audit. Hannah designs the programme's indicators with that standard of evidence in mind.
Lab — assurance indicators
"""Quarterly assurance indicators for app releases (synthetic)."""
q = {"Q1": (14, 9, 22, 15), "Q2": (16, 13, 24, 21), "Q3": (15, 15, 25, 25)}
for k, (rel, prov, sdks, reviewed) in q.items():
print(f"{k}: provenance verified {prov}/{rel} = {100 * prov // rel}% SDKs reviewed {reviewed}/{sdks} = {100 * reviewed // sdks}%")
Run with python3 lab_assurance.py. Verified output:
Q1: provenance verified 9/14 = 64% SDKs reviewed 15/22 = 68%
Q2: provenance verified 13/16 = 81% SDKs reviewed 21/24 = 87%
Q3: provenance verified 15/15 = 100% SDKs reviewed 25/25 = 100%
9.3 Lab result
Provenance-verified releases rose from 9 of 14 (64%) in Q1 to 13 of 16 (81%) in Q2 and 15 of 15 (100%) in Q3. SDK updates reviewed rose from 15 of 22 (68%) to 21 of 24 (87%) and 25 of 25 (100%).
The Q1 provenance gap came from the legacy build path described below, and the review gap came from apps that had not yet enabled the diff. Both figures reached 100% by Q3, but the totals are small, so one missed release would show clearly. Hannah shows the board the counts beside the percentages for that reason, and explains each quarter's misses by name.
9.4 Indicators
Indicators are chosen because they link to the February incident and SLSA-style build integrity. Each has a definition, data source and owner.
Hannah avoids indicators that are easy to count but say little, such as the number of policies written. She documents each indicator's definition on one page, so a new owner can produce the same figure next year. Data is drawn from build systems and pull requests rather than self-reported spreadsheets. Owners confirm each quarter's figures before they go to the CISO and the board.
9.5 Independent checks
Internal audit samples five releases each quarter and re-verifies provenance and review records.
Auditors pick the releases themselves, without notice, and rerun the provenance check using the published verification steps. They also confirm that each SDK update in those releases has a review record with a named reviewer and a date. Findings go to the CISO and to Hannah, with an agreed date for each fix. The Q3 sample led to the reviewer-name field becoming mandatory in the review template.
9.6 Gaps
In Q1, five releases lacked verified provenance because a legacy build path was still in use; it was retired in Q2.
Hannah reported the gap rather than waiting for it to close, because hiding it would have weakened trust in the other figures. The legacy path's owner agreed a retirement date with the platform team, and releases using it were listed until then. Reporting the gap also explained the Q1 figure to the board without anyone having to ask. The legacy app's later exception follows the same open approach.
9.7 Regulator
Northbridge's regulator receives a summary showing the February incident's follow-up and the indicators.
The summary covers what happened in February, the controls introduced since, and the quarterly indicators with their definitions. It states the remaining limits, such as the legacy app with an exception, rather than presenting the programme as finished. Hannah prepares it with the compliance team, which manages the bank's contact with the regulator, and it is sent alongside the regular reporting cycle. Questions from the regulator are answered within the agreed timeframe.
Reporting format
The board receives one page each quarter: three indicators with trends, the main risks, decisions needed and audit findings. Hannah writes the page in plain language and presents it in five minutes, leaving time for questions. Board members said the trend lines were more useful than any single number, because they showed whether the programme was working.
Regulator dialogue
Hannah meets the regulator's supervisor twice a year to discuss progress.
Worked example
Internal audit's Q3 sample finds all five releases verified, but one SDK review record lacked the reviewer's name. The template is changed to make the field mandatory.
Synthetic case. Indicators and audits are synthetic.
Errors and limits
100% indicators can hide weak reviews; audit quality as well as completion.
Exercises
- EX041 — Programme goals. List the three programme goals. Rubric: 3 = verifiable releases; SDK updates reviewed; criticals in 14 days; 2 = two; 1 = one; 0 = none.
- EX042 — Lab trend. Give the provenance trend. Rubric: 3 = 64% → 81% → 100%; 2 = partial; 1 = vague; 0 = none.
- EX043 — Q1 gap cause. Why were Q1 releases unverified? Rubric: 3 = legacy build path; 2 = partial; 1 = vague; 0 = none.
- EX044 — Audit finding. What did internal audit find in Q3? Rubric: 3 = missing reviewer name; field made mandatory; 2 = partial; 1 = vague; 0 = none.
- EX045 — Assurance meaning. What is assurance? Rubric: 3 = justified confidence with evidence; 2 = partial; 1 = vague; 0 = none.
Synthesis
Lead with clear goals and give assurance through verified, audited indicators.
Continue studying — chapter 09 of 12
← Chapter 08 (draft) · Course chapter index · Chapter 10 (draft) →