Chapter 5 — Application and software supply chain security: tools and interpretation
Lesson map
Lesson 05 published. Map, example, EX021.
flowchart TD
N0["Learning outcomes"]
N1["5.1 Scanner types"]
N2["Lab — finding triage"]
N3["5.2 Lab result"]
N4["5.3 Reachability"]
N5["5.4 Checking findings"]
N6["5.5 Alert fatigue"]
N7["Explaining findings"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX021"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["5.1 Scanner types"]
N2["Lab — finding triage"]
N3["5.2 Lab result"]
N4["5.3 Reachability"]
N5["5.4 Checking findings"]
N6["5.5 Alert fatigue"]
N7["Explaining findings"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX021"]
[Learning outcomes]
|
v
[5.1 Scanner types]
|
v
[Lab — finding triage]
|
v
[5.2 Lab result]
|
v
[5.3 Reachability]
|
v
[5.4 Checking findings]
|
v
[5.5 Alert fatigue]
|
v
[Explaining findings]
|
v
[Worked example] --> [Exercise EX021]
Learning outcomes
You will (1) interpret output from code, dependency and secret scanners; (2) deduplicate and rank findings; (3) judge reachability; (4) avoid alert fatigue.
5.1 Scanner types
Static analysis (SAST) reads code for risky patterns; software composition analysis (SCA) matches dependencies to advisories; secret scanners look for credentials. Each produces findings with different accuracy. At Harbourline, each scanner runs on every TrackPort build and produces its own list. SAST may flag code that is safe in context, SCA may report a vulnerability in a function TrackPort never calls, and secret scanners may flag test placeholders. Priya treats all scanner output as candidates to check, not as confirmed findings. She records which scanner and version produced each finding, because a changed advisory database can make the same TrackPort code produce different results.
Lab — finding triage
"""Deduplicate and rank synthetic scanner findings by severity and reachability (stdlib only)."""
from collections import Counter
findings = [ # tool, rule, file, severity, reachable
("sast", "sql-concat", "orders.py", "high", True),
("sast", "sql-concat", "orders.py", "high", True),
("sca", "ADV-21", "lockfile", "critical", False),
("sca", "ADV-22", "lockfile", "high", True),
("sast", "weak-hash", "legacy_export.py", "medium", False),
("secrets", "api-key", "tests/fixtures.py", "high", False),
("sast", "missing-authz", "shipments.py", "high", True),
]
unique = list(dict.fromkeys(findings))
rank = {"critical": 4, "high": 3, "medium": 2, "low": 1}
ordered = sorted(unique, key=lambda f: (f[4], rank[f[3]]), reverse=True)
print(f"raw {len(findings)}, unique {len(unique)}")
for i, (tool, rule, path, sev, reach) in enumerate(ordered, 1):
print(f"{i}. {rule:13} {path:18} {sev:8} {'reachable' if reach else 'not reachable'}")
print("by severity:", dict(Counter(f[3] for f in unique)))
Run with python3 lab_triage.py. Verified output:
raw 7, unique 6
1. sql-concat orders.py high reachable
2. ADV-22 lockfile high reachable
3. missing-authz shipments.py high reachable
4. ADV-21 lockfile critical not reachable
5. api-key tests/fixtures.py high not reachable
6. weak-hash legacy_export.py medium not reachable
by severity: {'high': 4, 'critical': 1, 'medium': 1}
5.2 Lab result
Seven raw findings reduce to six unique ones (the sql-concat finding in orders.py appeared twice). Ranked by reachability then severity: sql-concat in orders.py, ADV-22 and missing-authz in shipments.py are reachable highs and come first. The critical ADV-21 ranks fourth because the affected function is not reachable from TrackPort's code. The api-key in tests/fixtures.py and weak-hash in legacy_export.py follow. The ranking reflects TrackPort's real risk rather than scanner labels alone. Reachable findings come first, because an attacker could use them through TrackPort's code. The test-fixture key and the weak hash in legacy_export.py are checked separately, since neither is reachable through TrackPort's customer-facing code.
5.3 Reachability
A vulnerable function that the application never calls is less urgent than a reachable one, but it is still fixed in normal updates because code changes can make it reachable later. ADV-21 is a good example: it is critical, but TrackPort's code does not call the affected function, so it ranks fourth rather than first. It still gets a target date and is fixed in the next dependency update. Priya records her reachability judgement with the evidence, so a later reviewer can check it. Without a call graph tool, reachability is judged by reading code, which she notes as a limitation.
5.4 Checking findings
Priya reads each high finding in context. The test fixture api-key is a synthetic placeholder, so the secret scanner gets an allow-list entry with a review date. Checking in context avoids sending developers false alarms. The api-key in tests/fixtures.py is not a real credential, so it needs no rotation, but the allow-list entry has a review date so it is not ignored forever. Priya also reads the reachable highs to confirm they apply to TrackPort's code, and adds a suggested fix for each before creating the ticket.
5.5 Alert fatigue
Johnson et al. (2013) found that developers avoid static analysis tools largely because of false positives and poor explanations. Priya sends only checked, ranked findings with explanations. The TrackPort team's response shows why this matters: given three checked high findings with suggested fixes instead of the raw scanner list, they fixed all three in one sprint and asked for the same format in future. Priya's explanations describe the issue, where it occurs, why it matters and how to fix it, so developers do not have to research each finding themselves.
Explaining findings
Each finding sent to developers states what the scanner saw, why it matters in TrackPort, and a suggested fix, in a few sentences, with a link to the relevant ASVS requirement.
Worked example
The TrackPort team receives three checked high findings with suggested fixes instead of the raw scanner list. They fix all three in the sprint, and the team lead asks for the same format for future scans.
Synthetic case. Findings are synthetic.
Errors and limits
Reachability analysis can be wrong; record why something was judged unreachable.
Exercises
- EX021 — Dedup. How many unique findings were there? Rubric: 3 = 6 of 7; 2 = partial; 1 = vague; 0 = none.
- EX022 — Top three. Which findings ranked first? Rubric: 3 = sql-concat, ADV-22, missing-authz; 2 = two; 1 = one; 0 = none.
- EX023 — Critical rank. Why did critical ADV-21 rank fourth? Rubric: 3 = not reachable; 2 = partial; 1 = vague; 0 = none.
- EX024 — Fixture key. How was the fixture api-key handled? Rubric: 3 = allow-list with review date; 2 = partial; 1 = vague; 0 = none.
- EX025 — Fatigue. What did Johnson et al. find? Rubric: 3 = false positives deter use; 2 = partial; 1 = vague; 0 = none.
Synthesis
Deduplicate, rank by reachability and send checked findings.
Continue studying — chapter 05 of 12
← Chapter 04 (draft) · Course chapter index · Chapter 06 (draft) →