Chapter 14 — Software engineering
Lesson map
Lesson 14 published. Map, example, EX040.
flowchart TD
N0["Learning outcomes"]
N1["14.1 Infrastructure as code"]
N2["14.2 Security smells"]
N3["14.3 Pipelines"]
N4["14.4 Small changes"]
N5["14.5 Environments"]
N6["14.6 Peer review"]
N7["14.7 Documentation"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX040"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["14.1 Infrastructure as code"]
N2["14.2 Security smells"]
N3["14.3 Pipelines"]
N4["14.4 Small changes"]
N5["14.5 Environments"]
N6["14.6 Peer review"]
N7["14.7 Documentation"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX040"]
[Learning outcomes]
|
v
[14.1 Infrastructure as code]
|
v
[14.2 Security smells]
|
v
[14.3 Pipelines]
|
v
[14.4 Small changes]
|
v
[14.5 Environments]
|
v
[14.6 Peer review]
|
v
[14.7 Documentation]
|
v
[Worked example] --> [Exercise EX040]
Learning outcomes
You will (1) apply software engineering practices to infrastructure and platform code; (2) use version control, review and pipelines; (3) manage configuration as code; (4) reduce change risk.
14.1 Infrastructure as code
Solent defines its cloud networks, clusters and storage in code templates, and Kubernetes workloads in manifests, all in version control. Every change is a reviewed pull request, giving history and accountability.
Defining infrastructure in code means the same template produces the same network or cluster every time, unlike manual console changes that drift. Reviews catch mistakes before they reach production. History shows who changed what and when, which helps investigations and audits. Console changes are blocked in production except through an emergency procedure that is logged and reviewed.
14.2 Security smells
Rahman et al. (2019) identified recurring security smells in infrastructure-as-code: hard-coded secrets, admin by default, empty passwords, unrestricted IP addresses, use of HTTP and weak cryptography. Elena adds checks for several of these to the pipeline.
A smell is a pattern that often indicates a problem, not proof of one. A hard-coded secret in a template is visible to anyone with repository access; an unrestricted address range opens a service to the world. Automated checks find these patterns quickly so reviewers can focus on design.
14.3 Pipelines
A pipeline validates templates, runs policy checks, builds and scans images, and deploys to test before production. Failures stop the change.
Each stage catches a different kind of problem: template validation finds syntax errors, policy checks find insecure settings, image scans find known vulnerabilities and embedded secrets. Deploying to test first shows whether the change works. Stopping on failure keeps unsafe changes out of production. Each failure message names the stage and the rule that failed, so developers can fix problems without asking the platform team for help.
14.4 Small changes
Small, frequent changes are easier to review and roll back than large ones.
A change touching one service and two settings can be reviewed in minutes, and if it breaks something the cause is obvious. A change touching twenty services makes review superficial and diagnosis slow. Small changes also make rollback safer, since undoing one does not undo unrelated work. Solent's guideline asks for changes that a reviewer can understand in fifteen minutes; larger work is split into a series of steps, each deployable on its own.
14.5 Environments
Test, staging and production are separate accounts with separate credentials; promotion uses the same artifacts.
Separate accounts mean a mistake or compromise in test cannot reach production data. Promoting the same artifacts, rather than rebuilding them, ensures that what was tested is what runs in production. Credentials for each environment are held by different people or systems. Configuration that differs between environments, such as hostnames and sizes, lives in separate parameter files reviewed like code, so the differences are visible and deliberate rather than hidden in manual steps.
14.6 Peer review
Reviewers check intent, security settings and blast radius: what else could this change affect?
Intent asks whether the change does what its description says. Security settings are checked against policy. Blast radius asks what else could break, such as a network rule that also affects a monitoring service. Reviewers who ask these questions consistently catch problems that tests alone miss. Solent requires one reviewer for routine changes and two for changes to network policies, access rules or the pipeline itself, since mistakes there have the widest effect.
14.7 Documentation
README files in each repository explain how to deploy and roll back.
A deployment that only one person knows how to run or undo is a risk when that person is away. README files give the steps, prerequisites and how to check success. Rollback instructions are tested, so they work when needed under pressure. Each README also lists the owner and an escalation contact. Elena tests two READMEs by following them on the test cluster without help, and fixes the steps that were missing or out of date.
Console changes
Because console changes bypass the pipeline, Solent limits console write access in production to a break-glass role used only in emergencies, with alerts on use. Normal changes go through code.
Rollback design
Each deployment keeps the previous version ready, so rollback is one reviewed command; Elena tests rollback in staging monthly.
Merge rules
Main branches require one approval and passing checks before merge.
Worked example
The new IaC checks flag a storage template that allows access from any IP address, used by three projects. The template is fixed once and the three projects pick up the fix on their next deployment, an example of why fixing shared templates is efficient.
Synthetic case. Templates and findings are synthetic.
Errors and limits
Pipeline checks only see what passes through the pipeline; console changes bypass them.
Exercises
- EX040 — Smells. Name three IaC security smells. Rubric: 3 = hard-coded secrets, admin default, unrestricted IP, HTTP, weak crypto; 2 = two; 1 = one; 0 = none.
- EX041 — Blast radius. What does blast radius mean in review? Rubric: 3 = what else a change affects; 2 = partial; 1 = vague; 0 = none.
- EX042 — Template fix. Why was the template fix efficient? Rubric: 3 = three projects fixed at once; 2 = partial; 1 = vague; 0 = none.
Synthesis
Treat infrastructure like software: reviewed, versioned and tested.
Continue studying — chapter 14 of 50
← Chapter 13 (draft) · Course chapter index · Chapter 15 (draft) →