Chapter 40 — Verification laboratory II
Lesson map
Lesson 40 published. Map, example, EX118.
flowchart TD
N0["Learning outcomes"]
N1["40.1 Authorisation"]
N2["40.2 WSTG structure"]
N3["40.3 Pipeline review"]
N4["40.4 Evidence"]
N5["40.5 Results"]
N6["40.6 Reporting"]
N7["Retesting"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX118"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["40.1 Authorisation"]
N2["40.2 WSTG structure"]
N3["40.3 Pipeline review"]
N4["40.4 Evidence"]
N5["40.5 Results"]
N6["40.6 Reporting"]
N7["Retesting"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX118"]
[Learning outcomes]
|
v
[40.1 Authorisation]
|
v
[40.2 WSTG structure]
|
v
[40.3 Pipeline review]
|
v
[40.4 Evidence]
|
v
[40.5 Results]
|
v
[40.6 Reporting]
|
v
[Retesting]
|
v
[Worked example] --> [Exercise EX118]
Learning outcomes
You will (1) plan an authorised security test of the application and pipeline; (2) use the OWASP WSTG to structure testing; (3) verify ASVS requirements with evidence; (4) report results responsibly.
40.1 Authorisation
Testing requires written authorisation stating scope (staging ticket-web and pipeline), time window and contacts. Kwame's team tests only within that scope.
Written scope protects everyone. Without it, a test that slows the staging site or touches the wrong system could be treated as an attack, and the testers could face disciplinary or legal consequences. The time window lets operations recognise test traffic, and named contacts mean a problem can be stopped quickly. Production is excluded because riders depend on it. If the team finds something that seems to reach outside scope, such as a link to a vendor system, it stops and asks rather than following it.
40.2 WSTG structure
The OWASP Web Security Testing Guide v4.2 organises tests into areas such as configuration, identity, authentication, authorisation, session management, input validation and business logic. The team selects tests matching their ASVS Level 2 requirements.
Selecting tests from the WSTG by matching them to chosen requirements keeps the effort focused. A test for business logic, for example, checks refund limits that no generic scanner understands. Each selected test names the requirement it covers, so results feed straight into the pass or fail table. Areas with no matching requirement are noted, in case the requirement set itself has a gap.
40.3 Pipeline review
For the pipeline, the team reviews configuration: branch protection, secret exposure to forks, runner isolation, policy enforcement mode.
These settings decide whether the pipeline controls actually hold. Branch protection that administrators can bypass, secrets visible to fork builds or a policy left in warn mode would each undo much of the design. The review compares the live settings with the intended ones and records any differences. Configuration is read through the CI system's own settings pages and exports, so no change is made during review, and findings are confirmed by a second team member before being reported.
40.4 Evidence
Each ASVS requirement gets pass, fail or not applicable, with evidence such as screenshots of configuration or test logs.
Evidence lets someone else confirm the result without repeating the test. A pass with no evidence is a claim; a pass with a log showing the refused request is a fact. Not applicable is recorded with a reason, such as a feature ticket-web does not offer, so reviewers can challenge it if they disagree. Evidence is stored with the report and referenced by requirement identifier, so a later reader can find each item quickly. Screenshots are checked for personal data before being saved.
40.5 Results
30 application requirements: 27 pass, 2 fail (session timeout too long; missing rate limit on password reset), 1 not applicable. 12 pipeline requirements: 11 pass, 1 fail (a self-hosted runner still used for a reporting job).
The failures are specific and fixable. A long session timeout leaves accounts open on shared devices; a missing rate limit on password reset lets someone flood riders with reset messages or guess codes. The self-hosted runner breaks the requirement that builds use hosted, isolated runners, even though the job only produces reports.
40.6 Reporting
Findings describe the weakness, its impact and the fix, without step-by-step misuse instructions, and go to owners with target dates.
Detailed misuse steps help attackers if the report leaks and add nothing for owners, who need to know what to fix. Each finding states the requirement it fails, the evidence, the likely impact on riders or Coastline, and a suggested fix. Target dates follow the remediation targets by severity. Reports go to the named owners and Ingrid, not to a broad mailing list, and are stored with restricted access until every finding is fixed.
Retesting
Each fix is retested by someone other than the person who made it, using the same test as the original finding. Retest evidence is attached to the finding before it is closed. Kwame finds that one fix for the rate limit applied only to the web form and not the mobile API; the retest caught it and the fix was extended.
Lessons for the course
The team notes which tests found real issues to plan next year's verification focus.
Sharing results
Results are shared with developers in a short session.
Worked example
All three failures are fixed within 30 days and retested. The verification report becomes evidence for Coastline's annual security review.
Synthetic case. Tests and results are synthetic.
Errors and limits
A test window samples the system at one time; continuous checks fill the gaps.
Exercises
- EX118 — Authorisation contents. What must test authorisation state? Rubric: 3 = scope, time window, contacts; 2 = two; 1 = one; 0 = none.
- EX119 — Results. Summarise application and pipeline results. Rubric: 3 = app 27/2/1; pipeline 11/1; 2 = partial; 1 = one; 0 = none.
- EX120 — Pipeline failure. What pipeline failure was found? Rubric: 3 = self-hosted runner for reporting job; 2 = partial; 1 = vague; 0 = none.
Synthesis
Authorised, structured verification with evidence closes the loop on requirements.
Continue studying — chapter 40 of 50
← Chapter 39 (draft) · Course chapter index · Chapter 41 (draft) →