Chapter 4 — Scripting and data formats
Lesson map
Lesson 04 published. Map, example, EX013.
flowchart TD
N0["Learning outcomes"]
N1["4.1 Data formats"]
N2["Lab — Checking a week of backup results"]
N3["4.2 A backup check"]
N4["4.3 Bad data"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX013"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["4.1 Data formats"]
N2["Lab — Checking a week of backup results"]
N3["4.2 A backup check"]
N4["4.3 Bad data"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX013"]
[Learning outcomes]
|
v
[4.1 Data formats]
|
v
[Lab — Checking a week of backup results]
|
v
[4.2 A backup check]
|
v
[4.3 Bad data]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX013]
Learning outcomes
You will (1) read CSV and JSON exports; (2) write a script to check backup results; (3) handle bad data safely.
4.1 Data formats
Linden's systems export data in different formats. The backup appliance writes a CSV file each night with one row per practice: date, practice name, files copied and status. The practice management system's audit export is JSON, a format of nested keys and values. Nadia learns to read both with Python's standard library. Daniel explains that understanding formats is a core technical skill because most security checks begin by reading some system's export and turning it into a clear answer. She keeps sample files of each format for testing her scripts.
Lab — Checking a week of backup results
# Check a week of nightly backup results (synthetic CSV rows: date, practice, files, status).
rows = """2026-03-01,Elmfield,4120,ok
2026-03-02,Elmfield,4133,ok
2026-03-03,Elmfield,0,disk full
2026-03-04,Elmfield,4150,ok
2026-03-05,Elmfield,4161,ok
2026-03-06,Elmfield,0,disk full
2026-03-07,Elmfield,0,disk full
2026-03-01,Norbury,2210,ok
2026-03-02,Norbury,2215,ok
2026-03-03,Norbury,2219,ok
2026-03-04,Norbury,2224,ok
2026-03-05,Norbury,2230,ok
2026-03-06,Norbury,2233,ok
2026-03-07,Norbury,2241,ok
2026-03-07,Westcott,3100""".splitlines()
summary, unreadable = {}, 0
for r in rows:
f = r.split(",")
if len(f) != 4: unreadable += 1; continue
d, p, _, s = f
ok, n, last = summary.get(p, (0, 0, "never"))
summary[p] = (ok + (s == "ok"), n + 1, d if s == "ok" else last)
for p, (ok, n, last) in sorted(summary.items()):
print(f"{p}: {ok}/{n} nights ok, last success {last}" + (" <-- CHECK" if ok < n else ""))
print(f"unreadable rows: {unreadable}")
Run with python3 lab_backups.py. Verified output:
Elmfield: 4/7 nights ok, last success 2026-03-05 <-- CHECK
Norbury: 7/7 nights ok, last success 2026-03-07
unreadable rows: 1
4.2 A backup check
Nadia writes a script that reads a week of synthetic backup results and reports, for each practice, how many nights succeeded and the last successful date. The lab shows that the Elmfield practice failed on three nights, including the last two. Grace had not noticed because the appliance only emails on success. Nadia's script turns a quiet failure into a visible one. Grace finds a full disk on the Elmfield imaging server and clears space, and the next night's backup succeeds.
4.3 Bad data
Daniel asks what happens if a line is missing a field or has a misspelt status. Nadia adds checks: rows with missing fields are reported as unreadable rather than ignored, and any status other than "ok" counts as a failure. Treating unknown values as failures is safer than treating them as success, because a monitoring script that hides problems is worse than none. She also records the script's expected format at the top of the file so future users know what it assumes.
Worked example
Nadia reads CSV and JSON exports, writes a backup check that finds three failures at Elmfield and handles bad rows safely.
Synthetic case. Practices and results are synthetic.
Errors and limits
Scripts break when export formats change. Success emails create false comfort.
Exercises
- EX013 — Quiet failure. Why had Grace missed the Elmfield failures? Rubric: 3 = the appliance only emailed on success;; 2 = success emails;; 1 = vague;; 0 = none.
- EX014 — Lab result. What did the lab report for Elmfield? Rubric: 3 = 4 of 7 nights ok, last success 2026-03-05;; 2 = three failures;; 1 = vague;; 0 = none.
- EX015 — Unknown status. Why treat unknown status values as failures? Rubric: 3 = hiding problems is worse than no monitoring;; 2 = safe default;; 1 = success;; 0 = none.
- EX016 — Root cause. What caused the Elmfield failures? Rubric: 3 = a full disk on the imaging server;; 2 = disk;; 1 = network;; 0 = none.
Synthesis
Reading exports with small scripts turns quiet failures into visible ones, provided bad data is treated cautiously.
Continue studying — chapter 04 of 18
← Chapter 03 (draft) · Course chapter index · Chapter 05 (draft) →