Chapter 35 — Design studio II
Lesson map
Lesson 35 published. Map, example, EX103.
flowchart TD
N0["Learning outcomes"]
N1["35.1 The brief"]
N2["35.2 Alternatives"]
N3["35.3 Trade-offs"]
N4["35.4 Exceptions life cycle"]
N5["35.5 Package"]
N6["35.6 Critique"]
N7["Assessment"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX103"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["35.1 The brief"]
N2["35.2 Alternatives"]
N3["35.3 Trade-offs"]
N4["35.4 Exceptions life cycle"]
N5["35.5 Package"]
N6["35.6 Critique"]
N7["Assessment"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX103"]
[Learning outcomes]
|
v
[35.1 The brief]
|
v
[35.2 Alternatives]
|
v
[35.3 Trade-offs]
|
v
[35.4 Exceptions life cycle]
|
v
[35.5 Package]
|
v
[35.6 Critique]
|
v
[Assessment]
|
v
[Worked example] --> [Exercise EX103]
Learning outcomes
You will (1) design monitoring and response for configuration drift and policy exceptions; (2) critique designs in teams; (3) produce an operational design package.
35.1 The brief
Design how Solent detects and responds to differences between declared and running configuration, and manages exceptions through their life.
Drift between declared and running configuration may come from an emergency fix, a mistake or an attacker. Exceptions, if unmanaged, become permanent weaknesses. The brief asks teams to handle both, since they share a theme: keeping reality aligned with what was approved. Teams must show how their design would handle a manual fix made during an incident at night. Each team has two weeks.
35.2 Alternatives
Team A proposes relying on GitOps reconciliation to revert drift silently. Team B proposes reconciliation plus alerts for security-relevant drift. Elena's team proposes B plus a weekly drift report covering resources outside GitOps, such as cloud accounts.
Each option goes further than the last. Silent reversion fixes drift but leaves no trace. Alerts add visibility for important changes. A weekly report extends coverage to resources the controller does not manage. Teams compare the options on visibility, noise and effort. Costs are estimated for each.
35.3 Trade-offs
Silent reversion hides useful signals: someone made a manual change, perhaps during an incident. Alerts on every drift create noise. Elena's team limits alerts to security-relevant fields.
A manual change during an incident may be legitimate but should still be noticed and reviewed. Alerting on every field would overwhelm the on-call engineer with trivial differences. Focusing on security-relevant fields, such as privilege settings, balances visibility and noise. The team lists the security-relevant fields explicitly, such as privileged, capabilities and network policy rules, and reviews the list each quarter.
35.4 Exceptions life cycle
Request, review, approval, labelled implementation, expiry reminder, renewal or removal, all logged.
Each stage has a clear owner and record. Review checks the reason and compensating controls; approval makes someone accountable; labels let the policy engine enforce expiry; reminders give time to plan renewal or removal. Logging every step creates an audit trail. The policy engine enforces expiry, and the workflow tool sends reminders at thirty and seven days. Renewal needs the same review as the original request, so an exception cannot roll over unexamined. Removal is confirmed by checking that the label has gone and the workload still runs.
35.5 Package
Diagrams, alert list with severities, exception workflow and test plan.
The alert list shows which drifts matter most and who responds. The exception workflow shows how requests move through stages. The test plan shows how the design will be verified, such as creating synthetic drift and checking that alerts fire. Elena's team keeps the package short, with the alert list as a table of field, severity, owner and response. Critics receive it in advance. The test plan includes one case for each alert and one for an exception expiring on schedule.
35.6 Critique
Critics ask what happens to drift in resources the GitOps controller does not manage; Elena's weekly report addresses this.
Cloud accounts, DNS settings and some managed services sit outside the GitOps controller's reach. Without the weekly report, drift there would go unnoticed. The critique confirms the value of the extra coverage and prompts Elena's team to list exactly which resources the report covers. Critics also ask how often the report is read; the team assigns a named reviewer and records each review.
Assessment
Teams are assessed on whether their design meets the brief, the quality of trade-off reasoning, the test plan and the response to critique. Elena's team loses a point for an alert list without clear owners, which they fix in the final package. The studio reinforces that operational design deserves the same rigour as building controls. Each team also writes a one-paragraph reflection on what they would change after critique, which the tutor reads alongside the package.
Prototype
Elena's team builds a small prototype of the weekly report on synthetic data to show it is feasible before the panel.
Worked example
The panel adopts Elena's team's design, noting that the weekly report would have caught the opening case's missing network policies, since they were declared in a design document but never in code.
Synthetic case. Designs are synthetic.
Errors and limits
Drift detection needs a trustworthy declared state; keep the repository protected.
Exercises
- EX103 — Silent reversion. What is the problem with silent reversion? Rubric: 3 = hides manual change signals; 2 = partial; 1 = vague; 0 = none.
- EX104 — Exception steps. List the exception life cycle steps. Rubric: 3 = request, review, approve, implement, remind, renew/remove; 2 = four; 1 = two; 0 = none.
- EX105 — Opening case link. How would the design have caught the opening case? Rubric: 3 = weekly report vs declared design; 2 = partial; 1 = vague; 0 = none.
Synthesis
Detect drift, alert on what matters and manage exceptions through their life.
Continue studying — chapter 35 of 50
← Chapter 34 (draft) · Course chapter index · Chapter 36 (draft) →