Chapter 30 — Verification laboratory I
Lesson map
Lesson 30 published. Map, example, EX088.
flowchart TD
N0["Learning outcomes"]
N1["30.1 Why test security rules"]
N2["Lab — Lab — refund authorisation tests"]
N3["30.2 Lab result"]
N4["30.3 Negative cases"]
N5["30.4 In CI"]
N6["30.5 Mutation thinking"]
N7["Test naming"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX088"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["30.1 Why test security rules"]
N2["Lab — Lab — refund authorisation tests"]
N3["30.2 Lab result"]
N4["30.3 Negative cases"]
N5["30.4 In CI"]
N6["30.5 Mutation thinking"]
N7["Test naming"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX088"]
[Learning outcomes]
|
v
[30.1 Why test security rules]
|
v
[Lab — Lab — refund authorisation tests]
|
v
[30.2 Lab result]
|
v
[30.3 Negative cases]
|
v
[30.4 In CI]
|
v
[30.5 Mutation thinking]
|
v
[Test naming]
|
v
[Worked example] --> [Exercise EX088]
Learning outcomes
You will (1) write unit tests for security rules; (2) test boundaries and negative cases; (3) run tests in CI; (4) interpret test results.
30.1 Why test security rules
Security rules, such as who may issue a refund, are business logic that can break silently. Tests make the intended behaviour explicit and catch regressions.
Refund rules mix role and amount, and a small edit can change who gets money back. Without tests, nobody notices until a fraud report or an audit. With tests, the change fails in CI within minutes and the author sees exactly which rule they altered. Tests also document the rule in a form developers read: the test names say what is allowed far more precisely than a policy document.
Lab — Lab — refund authorisation tests
"""Authorisation unit tests for a synthetic fare-refund function."""
import io, unittest
def can_refund(role, amount):
return role == "agent" and amount <= 50 or role == "supervisor"
class T(unittest.TestCase):
def test_rider_cannot(self): self.assertFalse(can_refund("rider", 5))
def test_agent_small(self): self.assertTrue(can_refund("agent", 50))
def test_agent_large(self): self.assertFalse(can_refund("agent", 51))
def test_supervisor(self): self.assertTrue(can_refund("supervisor", 500))
r = unittest.TextTestRunner(stream=io.StringIO()).run(unittest.defaultTestLoader.loadTestsFromTestCase(T))
print("tests run:", r.testsRun, "failures:", len(r.failures), "errors:", len(r.errors))
Run with python3 lab_unittest.py. Verified output:
tests run: 4 failures: 0 errors: 0
30.2 Lab result
Four tests run and all pass: a rider cannot refund, an agent can refund 50 but not 51, a supervisor can refund 500. The 50/51 pair tests the boundary exactly.
The test runner's output is captured in a string buffer, so the lab prints only the summary line. A real CI run shows each test name, which helps when one fails. The function relies on Python's operator precedence, where 'and' binds more tightly than 'or'; the tests confirm that the expression means what the author intended. Adding parentheses would make that intent clearer to a reader.
30.3 Negative cases
Tests that check something is refused are as important as tests that check something is allowed. Two of the four lab tests are negative.
Many authorisation bugs are about allowing too much, and positive tests never catch them: a function that returns True for everyone passes every 'is allowed' test. The rider test and the agent-large test would both fail in that case. Kwame aims for at least one refused case for every allowed one, covering wrong role, wrong amount and missing values.
30.4 In CI
The tests run on every change; a failure blocks merging. Coverage reports show which authorisation branches are exercised.
Running on every change means a regression is caught by the person who caused it, while the change is fresh in their mind. Blocking merges keeps broken rules out of the main branch. Coverage reports show branches no test reaches; for refunds, an uncovered branch is a rule nobody has checked. Coverage alone does not prove tests are good, which is why mutation thinking follows.
30.5 Mutation thinking
Kwame changes "<= 50" to "< 50" in a scratch copy: the agent-small test fails, proving the test detects that mistake.
A test suite that still passes after a deliberate mistake would not have protected anyone. Changing the comparison is a small mutation, the kind of slip a developer might make while editing nearby code. If no test failed, Kwame would know the boundary was untested. Tools can automate this by making many small changes and reporting which ones survive the test suite; each survivor shows a gap. Kwame runs such a tool monthly on the authorisation module, because that is where a missed mistake would cost most.
Test naming
Each test name states the rule in plain words, such as test_agent_large. When a test fails, the name tells the developer which rule broke without opening the code. Kwame adds a short docstring to each test linking it to the ASVS access-control requirement it checks, so auditors can trace requirements to evidence and developers understand why the rule exists.
Test data
Tests use synthetic riders and amounts only, never copies of real data. A small factory function creates test users with chosen roles, which keeps tests short and readable.
Worked example
A later change adds a "senior agent" role with a 200 limit. The developer adds two tests (200 allowed, 201 refused) before writing the code, and the review checks that the old tests still pass.
Synthetic case. Code and tests are synthetic.
Errors and limits
Passing tests show the cases written work; they do not prove no other path exists.
Exercises
- EX088 — Lab results. What did the lab report? Rubric: 3 = 4 run, 0 failures, 0 errors; 2 = partial; 1 = vague; 0 = none.
- EX089 — Boundary pair. Which tests check the boundary? Rubric: 3 = agent 50 allowed, 51 refused; 2 = partial; 1 = vague; 0 = none.
- EX090 — Mutation check. What did changing <= to < show? Rubric: 3 = test catches it; 2 = partial; 1 = vague; 0 = none.
Synthesis
Test allowed and refused cases at the boundaries, in CI, and check the tests can fail.
Continue studying — chapter 30 of 50
← Chapter 29 (draft) · Course chapter index · Chapter 31 (draft) →