Chapter 17 — Control and feedback basics
Lesson map
Lesson 17 published. Map, example, EX049.
flowchart TD
N0["Learning outcomes"]
N1["17.1 Loop"]
N2["17.2 Stability"]
N3["17.3 Manual vs automatic"]
N4["17.2b Choosing the sampling interval"]
N5["17.3b Human-in-the-loop placement"]
N6["17.1b Set-points and drift"]
N7["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX049"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["17.1 Loop"]
N2["17.2 Stability"]
N3["17.3 Manual vs automatic"]
N4["17.2b Choosing the sampling interval"]
N5["17.3b Human-in-the-loop placement"]
N6["17.1b Set-points and drift"]
N7["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX049"]
[Learning outcomes]
|
v
[17.1 Loop]
|
v
[17.2 Stability]
|
v
[17.3 Manual vs automatic]
|
v
[17.2b Choosing the sampling interval]
|
v
[17.3b Human-in-the-loop placement]
|
v
[17.1b Set-points and drift]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX049]
Learning outcomes
You will (1) model a control loop around assessment tooling; (2) choose sensors and actuators; (3) avoid unstable feedback (alert storms).
17.1 Loop
Sensor: out-of-scope probe count per hour. Controller: compare to threshold 0. Actuator: TrackScope hard-deny and page the lead. Plant: the engagement. Without the sensor, you only discover scope breaks in the final report.
The zero threshold means any out-of-scope probe is already a breach of the promise, not a soft warning. Paging the lead closes the loop while the window is still open, when stop and notify still help. Without an hourly count, the final report becomes the first sensor, which is too late for the client. Sam graphs the count live next to the deny log so a spike is obvious.
17.2 Stability
If every medium finding pages the whole company, people silence the pager (alert fatigue). Severity-based routing and rate limits on notifications are the damping. The baserate lab (Chapter 7) is the same idea for detectors.
Damping keeps the signal useful. Medium findings go to the owning team queue; only stop conditions and critical safety events page everyone. Rate limits stop a flapping sensor from waking the company every minute. The same lesson as the base-rate work applies: a detector that cries often is turned off.
17.3 Manual vs automatic
Hard denies of deny-listed hosts can be automatic. Changing the ROE mid-engagement must stay manual and signed. Put automation on the inner loop; keep humans on the outer loop.
Synthetic case — Riverstone Transit. After the 11:10 deny-list touch, Sam added a sensor: any deny-list hit pages the lead within one minute. False pages in a week: zero.
Automatic deny is fast enough to stop a bad probe in the same second. A mid-window ROE change alters what is legal, so it needs a signature and two people. Drawing that line in the design review prevents arguments during the engagement. Sam's runbook says which class of change is which.
17.4 Errors and limits
Feedback assumes the sensor is honest. A broken logger looks like a quiet system. Heartbeat the sensors.
A silent logger can hide a flood of denies or pretend none occurred. Heartbeats prove the sensor still reports, even when the count stays at zero. Sam pages on missing heartbeats with the same urgency as a real spike, because both mean the loop is blind. He checks the heartbeat path in staging before each engagement window opens and keeps a screenshot of the green check in the dossier.
17.2b Choosing the sampling interval
A sensor that samples once an hour can miss a five-minute out-of-scope burst. Sample at least twice as often as the shortest stop-condition window you promised the client. If the kill-switch must act in 30 seconds, poll every 10–15 seconds. The cost is cheap compared with explaining a missed stop in the report.
Nyquist-style thinking applies: sample faster than the event you must catch. Ten to fifteen seconds leaves margin for slow disks and busy leaders. Sam records the chosen interval next to the stop-condition text so auditors see they match.
17.3b Human-in-the-loop placement
Put humans where judgment about authorization is required; put automation where delay is dangerous. Approving a scope widen is human. Denying a deny-listed host is automation. Mixing them — waiting for a human to approve a deny — recreates the incident the control was meant to stop.
Judgment about permission cannot be rushed into a script; delay on a deny can. Waiting for a human to approve a deny-list hit is how out-of-scope traffic continues. Sam's design review checklist asks of every control whether delay helps or hurts, and places the human only where it helps.
17.1b Set-points and drift
Write the set-point next to the sensor ("out-of-scope probes per hour = 0"). Review set-points when the client changes windows or when false pages exceed an agreed rate. Unreviewed set-points drift into ignored alarms.
A set-point nobody revisits becomes folklore. Client window changes and false-page budgets both force a fresh look. Writing the number beside the sensor keeps operators from inventing a private threshold. Sam reviews set-points at kickoff and after any week that exceeded the false-page budget, and he writes the new number into the sensor config in the same change.
Exercises
- EX049 — Loop design. Name sensor, threshold, actuator for SLA breaches. Rubric: 3 = all three coherent; 2 = two; 1 = one; 0 = none.
- EX050 — Fatigue. Give one damping rule for finding notifications. Rubric: 3 = specific rule (batch, severity, rate); 2 = vague "don't spam"; 1 = none useful; 0 = none.
- EX051 — Manual boundary. Which change must not be automated mid-engagement? Rubric: 3 = ROE/scope widening with reason; 2 = partial; 1 = wrong; 0 = none.
Synthesis
Sense, compare, act — with humans on authorization and machines on enforcement.
Continue studying — chapter 17 of 50
← Chapter 16 (draft) · Course chapter index · Chapter 18 (draft) →