Chapter 3 — Programming foundations
Lesson map
Lesson 03 published. Map, example, EX007.
flowchart TD
N0["Learning outcomes and connection"]
N1["3.1 Why a responder programs"]
N2["3.2 Core constructs you will use"]
N3["3.3 Evidence-safe scripting"]
N4["3.4 Testing before trusting"]
N5["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX007"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["3.1 Why a responder programs"]
N2["3.2 Core constructs you will use"]
N3["3.3 Evidence-safe scripting"]
N4["3.4 Testing before trusting"]
N5["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX007"]
[Learning outcomes and connection]
|
v
[3.1 Why a responder programs]
|
v
[3.2 Core constructs you will use]
|
v
[3.3 Evidence-safe scripting]
|
v
[3.4 Testing before trusting]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX007]
Learning outcomes and connection
You will be able to (1) write small, readable Python programs that parse, filter and summarize incident data; (2) make scripts safe for evidence (read-only, deterministic, logged); and (3) test a script against a known input before trusting its output. Chapters 1–2 described systems and paths; from here on you will automate the repetitive parts of the response service, and every automation starts as a small script.
3.1 Why a responder programs
Response work repeats: normalizing timestamps, extracting accounts from logs, counting events per source, building evidence manifests. Doing it by hand is slow and error-prone under pressure. A technologist writes scripts that a colleague can read, rerun and check. This book's labs use only the Python 3 standard library so they run offline on any workstation without installing packages — a deliberate choice for an evidence workstation. The failed sign-ins script in this chapter is an example: it counts failures per account and distinct sources per account, a job that is slow and error-prone by hand on a large export.
3.2 Core constructs you will use
Variables and types (strings, integers, lists, dictionaries); loops and comprehensions; functions with clear inputs and outputs; reading files line by line; the csv, json, datetime, hashlib, collections and statistics modules. Prefer explicit names (signin_events) over clever one-liners. These are enough for most response scripts. The failed sign-ins script uses only csv to read the export, a dictionary and set to count failures and sources per account, and sorting to make the output repeatable. A colleague reading it can check each line against the problem, which is the point of keeping scripts plain. When a script needs anything beyond the standard library, the case notes record the package and version.
3.3 Evidence-safe scripting
Rules: open evidence files read-only and write results to a separate folder; never modify originals; make output deterministic (sort before printing); record the script version (hash or commit) and the input file hashes in the case notes; fail loudly on malformed lines rather than silently skipping them, or count and report what was skipped. NIST SP 800-86 emphasizes preserving the integrity of data during examination; a script is part of the examination method. The failed sign-ins script follows these rules: it reports skipped rows instead of hiding them.
3.4 Testing before trusting
Before running a parser over a real export, run it over a tiny input you can verify by eye, including edge cases: an empty field, a timestamp without a zone, a duplicated line. Compare the output with what you expect. If they differ, fix the script or your expectation, then record why. The failed sign-ins script was tested this way. A ten-line file with one empty user field produced the expected "skipped 1", which showed that the script would report malformed rows rather than hide them. Only then was it run on the real export, and both the script hash and the input hash went into the case record.
3.5 Worked example (synthetic): counting failed sign-ins
Fictional. Fernmoor's identity provider exports sign-in events as CSV with columns time,user,result,src. The duty lead asks: which accounts had more than five failures in the export, and from how many sources? A minimal, evidence-safe approach:
read rows with csv.DictReader (file opened read-only)
for each row: if result == "failure": count[user] += 1; sources[user].add(src)
report users with count > 5, sorted by count descending, with len(sources)
report number of rows skipped for missing fields
Tested first on a ten-line file containing one row with an empty user, the script reported "skipped 1", which is correct. Run on the real export, it listed water-billing-2 with 37 failures from 29 sources — a pattern consistent with password spraying that the analyst then examined alongside successful sign-ins (Chapter 16). The script and input hashes were attached to the case.
3.6 Common errors and limits
- Silently dropping malformed lines, which hides exactly the anomalies you need.
- Mixing time zones when comparing timestamps.
- Editing evidence in a spreadsheet that reformats dates and strips leading zeros.
- Treating a script's output as fact without a test on known input.
Fernmoor's own work shows how easy these are to make. Opening an identity export in a spreadsheet can convert timestamps to the workstation's local zone without warning, and a script that silently drops rows with an empty user would have hidden the malformed line the test caught. Keeping inputs read-only and printing skipped counts guards against both.
Exercises — chapter 3
Attempt each exercise before opening the worked results. Every case, organization, person, address and number in an exercise is synthetic unless a cited public source is named.
EX007 — Pseudocode to Python. Steps: turn the pseudocode from the failed sign-ins worked example into a standard-library Python script. Rubric: 3 = uses csv, Counter/dict and set, sorts output, reports skipped rows, opens input read-only; 2 = works but no skipped count or not sorted; 1 = partial; 0 = none.
EX008 — Test input. Steps: design a ten-line test file and the exact expected output. Rubric: 3 = includes a user above and at the threshold, an empty field, duplicate sources, and states expected output; 2 = missing edge cases; 1 = no expected output; 0 = none.
EX009 — Evidence note. Steps: write the case-note entry recording how the script was used. Rubric: 3 = script identifier/hash, input file and hash, time, operator, result summary, skipped rows; 2 = missing hashes; 1 = result only; 0 = none.
Chapter synthesis
Write small, readable, standard-library scripts; keep evidence read-only and outputs deterministic; test on known input first; and record script and input hashes so another responder can reproduce your result.
Continue studying — chapter 03 of 30
← Chapter 02 · Course chapter index · Chapter 04 →
Continue to the next authored lesson.