Chapter 40 — Verification laboratory II

Chapter 40 — Verification laboratory II

Lesson map

Lesson 40 published. Map, example, EX118.

flowchart TD
  N0["Learning outcomes"]
  N1["40.1 Regression testing"]
  N2["40.2 Testing the tooling"]
  N3["40.3 Reporting regressions"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX118"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["40.1 Regression testing"]
  N2["40.2 Testing the tooling"]
  N3["40.3 Reporting regressions"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX118"]

[Learning outcomes]
    |
    v
[40.1 Regression testing]
    |
    v
[40.2 Testing the tooling]
    |
    v
[40.3 Reporting regressions]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX118]

Learning outcomes

You will (1) re-verify controls after changes; (2) test GRC tooling itself; (3) report regression findings.

40.1 Regression testing

After Ardwell upgrades its ticketing system, Nadia re-runs the verification tests for the GRC tooling and the broker. She learns that changes can break controls in unexpected ways. The upgrade changes how recurring tasks are scheduled, so several quarterly control tasks are now due annually. Without regression tests, the change would have silently reduced control frequency for a year. She adds a check that compares scheduled frequencies with the control register after every upgrade, closing that gap. Quarterly tasks becoming annual would also have weakened the access review evidence the auditors rely on.

40.2 Testing the tooling

The GRC tooling itself must be verified. Nadia tests that evidence attachments cannot be altered after closure, that only authorised staff can close findings and that reports count tasks correctly. One report counts reopened findings twice, inflating the open count that would reach the board. She fixes the query and adds a test case. She learns that tools that produce governance evidence deserve the same rigour as the controls they track, since errors in the tool become errors in the board's view of risk.

40.3 Reporting regressions

Nadia reports both regressions with severity, cause and fix, and recommends that upgrades to GRC systems follow change management with verification before release. She also proposes keeping a small library of test cases that run after any change. Prof. Brenner notes that the laboratory work shows a key principle: verification is not a one-time event but must be repeated whenever the system changes, because the conditions that made a control work can disappear quietly. The library of test cases makes repeating verification cheap enough that it will actually be done after each change.

Worked example

Nadia finds a schedule regression and a double-counting report error after an upgrade and builds regression tests for both.

Synthetic case. Tooling and defects are synthetic.

Errors and limits

Regression suites age. Not every change triggers tests.

Exercises

• EX118 — Schedule regression. What did the upgrade break? Rubric: 3 = quarterly tasks became annual;; 2 = frequency;; 1 = vague;; 0 = none.
• EX119 — Report defect. Which reporting error did Nadia find? Rubric: 3 = reopened findings counted twice;; 2 = double count;; 1 = vague;; 0 = none.
• EX120 — Verification principle. What principle does the laboratory illustrate? Rubric: 3 = verification must repeat whenever the system changes;; 2 = repeat;; 1 = one-time;; 0 = none.

Synthesis

Re-verification after change protects controls and the tools that report on them.

Continue studying — chapter 40 of 50

← Chapter 39 (draft) · Course chapter index · Chapter 41 (draft) →