Chapter 29 — Application and software supply chain security: operations
Lesson map
Lesson 29 published. Map, example, EX085.
flowchart TD
N0["Learning outcomes"]
N1["29.1 Monitoring"]
N2["29.2 Routine updates"]
N3["29.3 Advisory response"]
N4["29.4 Incident process"]
N5["29.5 Holidays"]
N6["29.6 Hand-over"]
N7["On-call guide"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX085"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["29.1 Monitoring"]
N2["29.2 Routine updates"]
N3["29.3 Advisory response"]
N4["29.4 Incident process"]
N5["29.5 Holidays"]
N6["29.6 Hand-over"]
N7["On-call guide"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX085"]
[Learning outcomes]
|
v
[29.1 Monitoring]
|
v
[29.2 Routine updates]
|
v
[29.3 Advisory response]
|
v
[29.4 Incident process]
|
v
[29.5 Holidays]
|
v
[29.6 Hand-over]
|
v
[On-call guide]
|
v
[Worked example] --> [Exercise EX085]
Learning outcomes
You will (1) operate the application securely; (2) monitor logs and alerts; (3) manage dependency updates routinely; (4) run a simple incident process.
29.1 Monitoring
CampusEats logs sign-ins, role changes, failed authorisation, payment callback verification failures and deployments. Alerts go to two on-call members during term.
Each alert has a short instruction saying what to check first. Lena tests the alerts each term by causing a harmless event, such as a failed callback signature in the test environment, and confirming that both on-call members receive it. Outside term, alerts go to one administrator, since ordering is closed. Logs needed for alerts are kept ninety days, as agreed earlier.
29.2 Routine updates
A bot opens weekly pull requests updating dependencies within allowed ranges; tests run automatically, and members merge them on Mondays. Major version updates are planned separately.
Merging on a fixed day spreads the work and keeps updates small, so a failure points to one or two packages rather than dozens. The owner of each dependency reviews its update. Major updates get their own branch, a test plan and a decision record, since they may change behaviour in ways the tests do not cover.
29.3 Advisory response
High-severity advisories affecting CampusEats are fixed within 7 days, medium within 30.
The deadlines run from the time the advisory is published, which matches the measure in the termly summary. If a fix is not available in time, the owner records an exception with a mitigation, such as disabling the affected feature, and an expiry date. Lena reports the share of advisories fixed within deadline each term, alongside the median fix time recorded in her earlier baseline. Low-severity advisories are handled in the weekly updates.
29.4 Incident process
Detect, assess, contain, fix, recover, review. A short template records each incident.
The template asks for a timeline, the impact, the actions taken and the lessons, in a page or less. Even small incidents are recorded, because patterns across several minor events often reveal a weakness that no single event shows. The review step follows the blameless approach used for the token leak, and its actions are tracked until they are done. Lena keeps the incident records together in the tracker, so the review at the end of each term can look across them.
29.5 Holidays
During holidays, ordering is closed and the app shows a notice; updates continue so the app is current when term starts.
Keeping updates running avoids a large backlog at the start of term, when the society is busiest and new members are joining. A reduced on-call arrangement covers the period, and advisory deadlines still apply. Lena checks after each holiday that no exceptions expired unnoticed and that the first lunchtime of term works as expected. Holiday notices tell students when ordering reopens.
29.6 Hand-over
Each year graduating members hand over roles and access is reviewed.
The handover includes the owner lists for dependencies and secrets, open exceptions and the runbook. Leaving members' access is removed the same week, and secrets they could read are rotated. Lena runs the review with the new committee so they understand each role before taking it on, which avoids the gaps that appeared when the build service had no owner. The review is recorded with the date and the names of the new owners.
On-call guide
The on-call guide lists each alert, what it means, first checks and who to call. It is short enough to read on a phone. New on-call members shadow an experienced member for one week before taking alerts alone, and the guide is updated whenever an alert causes confusion. During exam weeks, on-call shifts are shared so nobody carries alerts during their own exams.
Communication with users
When an incident affects ordering, a banner on the app tells students what is happening and when service is expected back. After incidents involving their data, students receive a short email explaining what happened and what was done, approved by the data protection officer.
Worked example
One Monday, a weekly update pull request fails tests because a date library changed its time zone handling. The team holds that update, opens an issue with the maintainers, and merges the others. The held update is fixed the next week.
Synthetic case. Operations are synthetic.
Errors and limits
Volunteer projects have limited cover; be honest with users about response times.
Exercises
- EX085 — Logged events. List three logged events. Rubric: 3 = sign-ins, role changes, failed authorisation, callback failures, deployments (three); 2 = two; 1 = one; 0 = none.
- EX086 — Fix targets. What are the fix targets? Rubric: 3 = high 7 days, medium 30; 2 = one; 1 = vague; 0 = none.
- EX087 — Failed update. How was the failing update handled? Rubric: 3 = held; issue opened; others merged; 2 = partial; 1 = vague; 0 = none.
Synthesis
Routine updates, monitoring and a simple incident process keep a small app safe.
Continue studying — chapter 29 of 40
← Chapter 28 (draft) · Course chapter index · Chapter 30 (draft) →