Chapter 7 — Security governance, risk and compliance: controls and verification

Lesson map

Lesson 07 published. Map, example, EX031.

flowchart TD
  N0["Learning outcomes"]
  N1["7.1 Sample sizes"]
  N2["Lab — Zero-deviation sample sizes for three tolerabl"]
  N3["7.2 Testing"]
  N4["7.3 Conclusions"]
  N5["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5
  N5 --> EX["Worked example"]
  EX --> Q["Exercise EX031"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["7.1 Sample sizes"]
  N2["Lab — Zero-deviation sample sizes for three tolerabl"]
  N3["7.2 Testing"]
  N4["7.3 Conclusions"]
  N5["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX031"]

[Learning outcomes]
    |
    v
[7.1 Sample sizes]
    |
    v
[Lab — Zero-deviation sample sizes for three tolerabl]
    |
    v
[7.2 Testing]
    |
    v
[7.3 Conclusions]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX031]

Learning outcomes

You will (1) choose sample sizes for control testing; (2) test a control's operation; (3) state conclusions with their limits.

7.1 Sample sizes

Lena needs to test whether Marston's new-payee authentication works for every payment. Testing all payments is impractical, so she samples. The lab computes the smallest sample with no deviations that supports each tolerable deviation rate with ninety-five percent confidence. For ten percent, twenty-nine items suffice; for five percent, fifty-nine; for three percent, ninety-nine. She learns that tighter conclusions require larger samples, and that the tolerable rate should reflect the control's importance. Fifty-nine is the size she will use for new-payee authentication in the next section.

Lab — Zero-deviation sample sizes for three tolerable rates

"""Smallest zero-deviation sample whose 95% upper bound meets a tolerable rate: (1-t)^n <= 0.05."""
from math import ceil, log
for t in (0.10, 0.05, 0.03):
    n = ceil(log(0.05)/log(1-t)); print(f"tolerable {t:.0%}: sample {n} with no deviations")

Run with python3 lab_n.py. Verified output:

tolerable 10%: sample 29 with no deviations
tolerable 5%: sample 59 with no deviations
tolerable 3%: sample 99 with no deviations

7.2 Testing

Because new-payee authentication protects member funds, Lena chooses a five percent tolerable rate and samples fifty-nine new-payee payments at random from the quarter. For each, she checks the authentication log shows a successful second factor before the payment. All fifty-nine show it. She records the sample method, the selected items, the checks and the results so another person could repeat the test. Mr. Reyes points out that random selection across the whole quarter matters, because a sample from one week could miss a period when the control failed.

7.3 Conclusions

Lena concludes that, with ninety-five percent confidence, the control failed in no more than five percent of new-payee payments during the quarter. She notes limits: the test covers the app channel only, not branch payments, and checks that authentication happened, not that it was resistant to all attacks. She learns that clear conclusions with stated limits are more valuable than broad claims that the control works. Branch payments would need a separate sample before any statement about all new-payee payments could be made.

Worked example

Lena computes 29, 59 and 99 as zero-deviation sample sizes, tests 59 payments with no deviations and states a bounded conclusion.

Synthetic case. Payments and results are synthetic.

Errors and limits

Random sampling misses targeted abuse. Logs may be incomplete.

Exercises

• EX031 — Five percent sample. How many items with no deviations support a 5% tolerable rate?
  Show marking criteria

  Rubric: 3 = fifty-nine;; 2 = 59;; 1 = 29;; 0 = none.
• EX032 — Three percent sample. How many are needed for 3%?
  Show marking criteria

  Rubric: 3 = ninety-nine;; 2 = 99;; 1 = vague;; 0 = none.
• EX033 — Rate choice. Why choose a 5% tolerable rate here?
  Show marking criteria

  Rubric: 3 = the control protects member funds;; 2 = importance;; 1 = vague;; 0 = none.
• EX034 — Check performed. What did Lena check for each payment?
  Show marking criteria

  Rubric: 3 = a successful second factor before payment in the log;; 2 = log check;; 1 = vague;; 0 = none.
• EX035 — Test limits. Name one limit of the conclusion.
  Show marking criteria

  Rubric: 3 = covers the app channel only;; 2 = scope;; 1 = vague;; 0 = none.

Synthesis

Sized samples support bounded, clearly limited conclusions about control operation.

Continue studying — chapter 07 of 12

← Chapter 06 (draft) · Course chapter index · Chapter 08 (draft) →