◇ OpenCyber Academy
Postgraduate · Chapter 10 · English

Chapter 10 — Research reading and critique

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 10 — Research reading and critique

Lesson map

Lesson 10 published. Map, example, EX046.

flowchart TD
  N0["Learning outcomes"]
  N1["10.1 Selected readings"]
  N2["10.2 Validity"]
  N3["10.3 Transferability"]
  N4["10.4 Synthesis"]
  N5["10.5 Open questions"]
  N6["10.6 Critique practice"]
  N7["Reading group practice"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX046"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["10.1 Selected readings"]
  N2["10.2 Validity"]
  N3["10.3 Transferability"]
  N4["10.4 Synthesis"]
  N5["10.5 Open questions"]
  N6["10.6 Critique practice"]
  N7["Reading group practice"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX046"]

[Learning outcomes]
    |
    v
[10.1 Selected readings]
    |
    v
[10.2 Validity]
    |
    v
[10.3 Transferability]
    |
    v
[10.4 Synthesis]
    |
    v
[10.5 Open questions]
    |
    v
[10.6 Critique practice]
    |
    v
[Reading group practice]
    |
    v
[Worked example] --> [Exercise EX046]

Learning outcomes

You will (1) read research on software supply chains critically; (2) assess validity and transferability to a bank; (3) synthesise findings into guidance; (4) identify open questions.

10.1 Selected readings

Hannah reads Ladisa et al. (2023), a taxonomy of attacks on open-source supply chains with safeguards; Zimmermann et al. (2019) on concentration of risk in npm; Wermke et al. (2022) on trust practices in open-source projects; Newman et al. (2022) on Sigstore; and Torres-Arias et al. (2019) on in-toto.

She chose them because they cover the threats Northbridge faces: compromised packages, concentrated maintainer risk, and the verification tools that might answer both. Each reading is linked to a decision in her programme, so the reading group has a practical reason to discuss it.

10.2 Validity

For each paper she asks: what data, what method, what claims, what threats to validity? The taxonomy is built from literature and expert input, valuable for structure but not for estimating frequency. Ecosystem studies measure one registry at one time.

She notes each answer in a short table for each paper, so the group can compare the strength of evidence across them. Where a paper's method cannot support a claim, she marks the claim as a hypothesis to test at Northbridge rather than a finding to apply.

10.3 Transferability

Northbridge uses several ecosystems and commercial SDKs, which open-source studies may not cover. Findings about maintainer concentration likely transfer; specific percentages may not.

Commercial SDKs are the biggest gap, because they come as binaries from companies whose practices the open-source studies did not examine. Hannah therefore uses the papers to frame questions for vendors rather than to estimate Northbridge's own risk. She notes the date of each study as well, since registries and maintainer behaviour change over time. Where she cites a figure, she states which ecosystem and year it comes from.

10.4 Synthesis

Guidance for Northbridge: reduce dependency count; prefer well-maintained components; verify integrity (signatures, provenance); monitor maintainer and behaviour changes; and require vendors to provide transparency equivalent to open-source practice.

Each item links back to a reading and to a control already in Northbridge's programme, or to a gap in it. Signing and provenance for internal libraries is the clearest gap, which is why the reading group later chose it for a pilot. Hannah shares the synthesis with the platform team, who own the build pipeline.

10.5 Open questions

How should banks verify closed-source SDKs that offer no provenance? How effective are behaviour diffs at catching subtle changes? Hannah lists these as research gaps.

The first matters most to Northbridge, because several of its most sensitive SDKs are closed-source and their vendors offer no provenance. The second matters because the pilot used planted changes, which may be easier to see than the subtle changes a vendor might make in practice. She shares the list with her tutor as possible topics for further study, and with vendors as questions.

10.6 Critique practice

She writes a one-page critique of each paper, shared with her team as a reading group.

Each critique states the paper's claim, the evidence offered, the main threat to validity and what Northbridge should take from it. Team members take turns presenting a paper and answering questions on it. Disagreements are recorded rather than settled, since several papers support more than one reasonable reading. The practice also gives junior staff experience in reading research carefully before citing it in their work.

Reading group practice

Each month one team member presents a paper in fifteen minutes using the same questions: data, method, claims, limits, transfer. The group then agrees one practical takeaway. Over a year this builds a shared understanding of evidence across engineers, privacy and security staff, who previously relied on vendor marketing.

Worked example

The reading group decides to pilot Sigstore-style verification for internal libraries shared between teams, inspired by Newman et al. (2022), since internal libraries had no signing at all.

Synthetic case. Reading group decisions are synthetic; cited papers are real.

Errors and limits

Applying research outside its context can mislead; pilot before adopting.

Exercises

Synthesis

Read critically, judge transferability and turn evidence into local guidance and open questions.

Continue studying — chapter 10 of 12

← Chapter 09 (draft) · Course chapter index · Chapter 11 (draft) →