Lesson map
Lesson 03 published. Map, example, EX007.
flowchart TD
N0["Learning outcomes and connection"]
N1["3.1 Why responders program"]
N2["3.2 Variables, types and control flow"]
N3["3.3 Functions"]
N4["3.4 Input validation"]
N5["3.5 Preserve originals"]
N6["3.6 Tests"]
N7["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX007"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["3.1 Why responders program"]
N2["3.2 Variables, types and control flow"]
N3["3.3 Functions"]
N4["3.4 Input validation"]
N5["3.5 Preserve originals"]
N6["3.6 Tests"]
N7["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX007"]
[Learning outcomes and connection]
|
v
[3.1 Why responders program]
|
v
[3.2 Variables, types and control flow]
|
v
[3.3 Functions]
|
v
[3.4 Input validation]
|
v
[3.5 Preserve originals]
|
v
[3.6 Tests]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX007]
Learning outcomes and connection
You will be able to (1) write a short Python program using variables, control flow and functions; (2) validate input records against explicit rules; (3) report rejected records without altering the original data; and (4) test your program with normal and malformed inputs. Chapter 2 showed that systems produce records whose meaning depends on context; here you learn to read such records mechanically and honestly. Chapter 8 extends these skills to parsing and provenance.
3.1 Why responders program
Incident data arrives as files: exports of sign-ins, case-intake spreadsheets, lists of affected hosts. Doing the same check by hand on 5,000 rows is slow and error-prone; a twenty-line program does it the same way every time and can be shown to a reviewer. The value is not speed alone but repeatability: anyone can rerun the program on the same input and get the same output. For that reason this book uses only the Python 3 standard library: there is nothing to install and no network access.
3.2 Variables, types and control flow
A variable names a value. Python values have types: strings ("H-07"), integers (42), floats (4.2), booleans (True), lists and dictionaries. Types matter in incident data because the same characters can mean different things: "0042" as a string keeps its leading zeros (an asset tag), but as an integer becomes 42. Control flow decides what runs: if/elif/else for choices, for loops to visit each record. A good habit is to make each decision visible — every branch either accepts the record or records why it was rejected.
3.3 Functions
A function packages a step with a name, inputs and an output. validate(row) that returns a list of problems is easier to test than the same logic buried inside a loop. Functions should not change their inputs unexpectedly; a validator that "fixes" a date while checking it has destroyed evidence that the date was malformed.
In Lab 03 the function validate(row) returns a list rather than printing messages. That choice lets the same function serve the loop that sorts rows into accepted and rejected and the assert tests at the bottom, without either caller parsing text. A function that returns data and leaves presentation to its caller is easier to reuse, and its behaviour can be checked by a reviewer without running the whole program.
3.4 Input validation
Validation compares a record to explicit rules: required fields present, values in an allowed set, dates in an agreed format. Write rules before writing code, in plain language: "case_id is required and matches HC- followed by four digits; severity is one of 1, 2, 3, 4; reported_at is an ISO 8601 date-time." A validator should reject and explain, not guess and continue. The rejected records are themselves information: if all rejected rows came from one intake channel, the problem is upstream.
Order the checks so that one failure does not hide another. Lab 03 checks every required field before testing formats, so a row missing two fields reports both instead of stopping at the first. Each message names the field and the rule it broke.
3.5 Preserve originals
The original file is evidence of what was received. Never overwrite it. Read it, write results to a new file or to the screen, and record a hash of the input (Chapter 17 explains hashes) so a reviewer can confirm which file you processed. In code, keep the raw row next to any normalized values.
In Lab 03 the input is a string inside the program, so preservation is easy; with a real export you would open the file read-only, compute its hash before and after the run, and write the accepted and rejected lists to new files whose names include the date. If the two hashes differ, stop and find out why before trusting any result produced from that file.
3.6 Tests
A test runs your function on an input whose correct answer you already know. Include at least: a valid record, a record missing each required field, a value just outside an allowed set, and a malformed date. Python's assert statement is enough for small tests.
Write the expected answer before running the test, not after; a test whose expected value was copied from the program's own output only proves that the program agrees with itself. Lab 03 asserts that HC-0101 has no problems and that HC-0102 lacks a reporter. Those two cases cover one valid and one invalid path; the exercises ask you to add the edges, such as a severity of zero or a case number with a stray space.
3.7 Worked example and lab (synthetic): case-intake validator
The CSV below is fictional. Save as lab03.py and run with python3 lab03.py.
# Lab 03 - validate synthetic case-intake records without changing them (stdlib only)
import csv, io, re
from datetime import datetime
RAW = """case_id,reported_at,reporter,severity,summary
HC-0101,2026-03-02T08:14:00,front-desk,2,Suspicious email with invoice link
HC-0102,2026-03-02T09:40:00,,3,Laptop reported lost on bus
HC-0103,02/03/2026 10:05,it-helpdesk,2,Password reset not requested by user
HC-104,2026-03-02T11:20:00,pharmacy,1,Printer shows odd message
HC-0105,2026-03-02T12:01:00,lab,5,Booking page slow
HC-0106,2026-03-02T13:30:00,front-desk,4,
"""
ID = re.compile(r"HC-\d{4}")
REQUIRED = ["case_id", "reported_at", "reporter", "severity", "summary"]
def validate(row):
problems = []
for f in REQUIRED:
if not row[f].strip():
problems.append(f"missing {f}")
if row["case_id"] and not ID.fullmatch(row["case_id"]):
problems.append("bad case_id")
if row["severity"] and row["severity"] not in {"1", "2", "3", "4"}:
problems.append("severity not in 1-4")
if row["reported_at"]:
try:
datetime.fromisoformat(row["reported_at"])
except ValueError:
problems.append("reported_at not ISO 8601")
return problems
rows = list(csv.DictReader(io.StringIO(RAW)))
accepted, rejected = [], []
for r in rows:
p = validate(r)
(rejected if p else accepted).append((r["case_id"], p))
print("accepted:", [c for c, _ in accepted])
for c, p in rejected:
print("rejected:", c, "->", "; ".join(p))
assert validate(rows[0]) == []
assert validate(rows[1]) == ["missing reporter"]
print("tests passed; original text unchanged:", RAW.count("\n"), "lines")
Expected output:
accepted: ['HC-0101']
rejected: HC-0102 -> missing reporter
rejected: HC-0103 -> reported_at not ISO 8601
rejected: HC-104 -> bad case_id
rejected: HC-0105 -> severity not in 1-4
rejected: HC-0106 -> missing summary
tests passed; original text unchanged: 7 lines
Interpretation: only one of six records passes. That is not a failure of the program; it is a finding about the intake process. Note that HC-0103's date "02/03/2026" is ambiguous (2 March or 3 February); the validator rightly refuses to guess.
Run the lab twice and compare outputs; they must match exactly, which is the repeatability promised earlier. Then deliberately break one rule, for example by allowing severity 5, and check that HC-0105 moves to the accepted list and that the first assertion still passes. Seeing a result change for a known reason is how you learn to trust the cases where it does not change.
3.8 Common errors and limits
- Silently "repairing" values during validation.
- Converting identifiers to numbers and losing leading zeros.
- Testing only the happy path.
-
Treating a validator as proof that values are true: a well-formed date can still be wrong.
-
Collecting only the first problem in a row, so that fixing it reveals another and the intake team is asked twice.
- Overwriting the input file with cleaned values, losing the record of what was actually received.
- Assuming the six Lab 03 rows represent all intake channels; a validator tuned to one sample may reject legitimate formats from a channel it has never seen.
Exercises — chapter 3
Attempt each exercise before opening the worked results. Every case, organization, person, address and number in an exercise is synthetic unless a cited public source is named.
EX007 — Extend the rules. Steps: add a rule to Lab 03 that reporter must be one of front-desk, it-helpdesk, pharmacy, lab; run it and report the new output. Rubric: 3 = rule added without changing other behaviour, correct output reported; 2 = rule works but output not reported; 1 = code does not run; 0 = none.
EX008 — Ambiguous dates. Steps: explain why the validator should not convert 02/03/2026 10:05 automatically; propose what the intake team should record instead and how to handle existing ambiguous rows. Rubric: 3 = explains day/month ambiguity and missing time zone, proposes ISO 8601 with offset, routes old rows to the reporter for confirmation; 2 = partial; 1 = proposes guessing; 0 = none.
EX009 — Test design. Steps: write three additional assert tests: a record with severity "0", a record with leading/trailing spaces in case_id, and a record with all fields empty. Predict each result before running. Rubric: 3 = tests correct with predictions, and spaces case explained (" HC-0101" fails the pattern because fullmatch sees the space); 2 = one test wrong; 1 = no predictions; 0 = none.
Chapter synthesis
Write the rules in words, encode each as a visible check, reject and explain rather than guess, keep the original untouched and test the edges. Rejected records describe the intake process as much as the incident.