OpenCyber Academy
Postgraduate · Chapter 3 · English
Course code: OCA-152Chapter 03: OCA-152-C003

Chapter 3 — Application and software supply chain security: specialist evidence

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 03 published. Map, example, EX011.

flowchart TD
  N0["Learning outcomes"]
  N1["3.1 Too many findings"]
  N2["3.2 Severity and likelihood"]
  N3["3.3 Reachability"]
  N4["Lab — two rankings"]
  N5["3.4 Lab result"]
  N6["3.5 Ecosystem evidence"]
  N7["3.6 Explaining to engineers"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX011"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["3.1 Too many findings"]
  N2["3.2 Severity and likelihood"]
  N3["3.3 Reachability"]
  N4["Lab — two rankings"]
  N5["3.4 Lab result"]
  N6["3.5 Ecosystem evidence"]
  N7["3.6 Explaining to engineers"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX011"]

[Learning outcomes]
    |
    v
[3.1 Too many findings]
    |
    v
[3.2 Severity and likelihood]
    |
    v
[3.3 Reachability]
    |
    v
[Lab — two rankings]
    |
    v
[3.4 Lab result]
    |
    v
[3.5 Ecosystem evidence]
    |
    v
[3.6 Explaining to engineers]
    |
    v
[Worked example] --> [Exercise EX011]

Learning outcomes

You will (1) use specialist evidence to prioritise vulnerabilities in dependencies; (2) combine severity, exploitation likelihood and reachability; (3) read research on ecosystem risk; (4) explain prioritisation to engineers.

3.1 Too many findings

Northbridge's apps have hundreds of dependency findings each quarter. Treating all high-severity findings as urgent overwhelms teams.

In a typical quarter the mobile app alone reports more than two hundred findings, most of them in transitive dependencies the team never chose directly. Engineers had learned to treat the list as noise, and real fixes were delayed while they triaged items that could not affect the app. Hannah wants a ranking that sends effort to the few findings that matter most, and that engineers will accept as fair rather than arbitrary.

3.2 Severity and likelihood

CVSS gives a severity score describing how bad a flaw could be. EPSS (Jacobs et al., 2021) estimates the probability that a vulnerability will be exploited in the next 30 days, using observed activity. Severity and likelihood answer different questions.

Hannah uses both, because a severe flaw that nobody exploits may wait while a moderate one under active exploitation needs fixing this week. EPSS scores change daily, so the ranking is refreshed each time it is used. Neither input is enough alone, which is why the lab combines them.

3.3 Reachability

A flaw matters most if the vulnerable function is reachable from the app's code. Static analysis can estimate reachability, though imperfectly.

Northbridge's tool marks a finding reachable when a call path exists from app code to the vulnerable function. It can miss calls made through reflection or configuration, so Hannah keeps unreachable findings on the list at reduced weight rather than dropping them. Engineers can ask for a manual check whenever they think the tool has got a call path wrong. The reachability judgement is stored with each finding, so later reviews can see why it was ranked low.

Lab — two rankings

"""Rank synthetic findings by CVSS alone and by CVSS weighted with EPSS and reachability."""
f = [("F1 xml-lib", 9.8, 0.02, False), ("F2 http-client", 7.5, 0.61, True), ("F3 image-sdk", 8.1, 0.09, True),
     ("F4 json-lib", 5.3, 0.33, True), ("F5 crypto-shim", 9.1, 0.01, True)]
print("by CVSS:      ", [x[0].split()[0] for x in sorted(f, key=lambda x: -x[1])])
score = lambda x: x[1] * x[2] * (1 if x[3] else 0.1)
print("by risk score:", [x[0].split()[0] for x in sorted(f, key=lambda x: -score(x))])

Run with python3 lab_prioritise.py. Verified output:

by CVSS:       ['F1', 'F5', 'F3', 'F2', 'F4']
by risk score: ['F2', 'F4', 'F3', 'F5', 'F1']

3.4 Lab result

By CVSS alone, the order is F1, F5, F3, F2, F4. Weighting CVSS by EPSS and reducing unreachable findings to one tenth gives F2, F4, F3, F5, F1. F1 (CVSS 9.8) falls to last because its EPSS is low and it is not reachable; F2 (7.5) rises to first because it is reachable and has EPSS 0.61.

Hannah shares the five-finding example with team leads before rolling out the method, so they can see how one finding falls and another rises. Hannah notes that the weighting is a judgement, not a law, and teams may question it.

3.5 Ecosystem evidence

Decan et al. (2018) showed vulnerabilities spread through dependency networks and fixes reach dependents slowly; Kula et al. (2018) found developers often do not update. These support investing in fast updating, not only prioritisation.

For Northbridge this means the ranking is only half the answer. Hannah also asks teams to shorten the time between a fixed version appearing and the app adopting it, because slow updating leaves known flaws in place however well they are ranked. Batched weekly updates with tests make this routine work rather than a special project.

3.6 Explaining to engineers

Engineers trust prioritisation they can understand. Hannah shows the three inputs for each finding and lets teams challenge reachability judgements.

Each finding in the weekly list shows its CVSS score, its EPSS probability and the reachability result, with a link to the call path when one exists. If a team shows the tool was wrong, the finding is re-ranked and the correction is noted, which also helps Hannah judge how far to trust the tool. Shared reasoning makes the ranking easier to defend when a deprioritised finding is questioned later.

Deadlines remain

Every finding keeps a deadline by severity, so low-ranked items are still fixed.

Worked example

Using the weighted ranking, Northbridge's mobile team fixes the top ten findings in two weeks rather than spreading effort over sixty. A later review finds none of the deprioritised findings were exploited during the quarter.

Synthetic case. Findings and scores are synthetic.

Errors and limits

EPSS and reachability are estimates; a low score is not a guarantee, so keep deadlines for all findings.

Exercises

Synthesis

Combine severity, likelihood and reachability, and explain the ranking openly.

Continue studying — chapter 03 of 12

← Chapter 02 (draft) · Course chapter index · Chapter 04 (draft) →

Worked answers

EX011. CVSS measures severity; EPSS estimates probability of exploitation in 30 days.

EX012. CVSS: F1, F5, F3, F2, F4; risk score: F2, F4, F3, F5, F1.

EX013. Its EPSS is low and it is not reachable.

EX014. That fast updating matters, since fixes propagate slowly and developers often do not update.

EX015. By showing the three inputs and letting teams challenge reachability.