Chapter 8 — Programming II
Lesson map
Lesson 08 published. Map, example, EX022.
flowchart TD
N0["Learning outcomes"]
N1["8.1 Modules and responsibilities"]
N2["8.2 JSON and files"]
N3["8.3 Packages and model files"]
N4["8.4 Adversarial tests"]
N5["8.6b Tests that guard the parser"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX022"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["8.1 Modules and responsibilities"]
N2["8.2 JSON and files"]
N3["8.3 Packages and model files"]
N4["8.4 Adversarial tests"]
N5["8.6b Tests that guard the parser"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX022"]
[Learning outcomes]
|
v
[8.1 Modules and responsibilities]
|
v
[8.2 JSON and files]
|
v
[8.3 Packages and model files]
|
v
[8.4 Adversarial tests]
|
v
[8.6b Tests that guard the parser]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX022]
Learning outcomes
You will (1) organise programs into modules and functions with clear responsibilities; (2) use JSON and files safely; (3) manage third-party packages and model files with integrity checks; and (4) write tests that include adversarial cases.
8.1 Modules and responsibilities
Split programs so each part has one job. For an LLM application: input handling, retrieval, model call, policy check, tool execution, logging. Keeping the policy check in its own small module means it can be reviewed and tested independently of the model.
The separation also limits damage: if the retrieval module is manipulated into returning a hostile document, the policy check still decides independently whether any resulting tool call is allowed. In Ask Lumen, the booking checks Rina wrote in the programming chapter live in such a module, with tests that run on every change. A reviewer can read the whole module in a few minutes.
8.2 JSON and files
Parse structured data with strict parsers (json.loads) and validate fields, types and ranges. Reject unknown fields. Never use eval on data. When the model is asked to output JSON, validate it the same way, because the model can produce malformed or manipulated output.
For Ask Lumen, the model is asked to return a booking as JSON with a clinic code, a date and a patient ID. The parser rejects any extra field, such as an instruction to cancel other appointments, and any value outside the allowed ranges. A malformed answer leads to a polite failure message for the patient, never to a partial action.
8.3 Packages and model files
Install packages from trusted sources with pinned versions. Model files are code-like risks: some serialisation formats can execute code when loaded. Verify a file's SHA-256 hash against a trusted manifest before loading; the lab detects tampering this way.
"""Lab H: integrity of model artefacts with SHA-256 manifests (stdlib only).
Creates synthetic 'artefact' bytes, records a manifest, then detects a
one-byte tampering and a swapped file. No real models are involved."""
import hashlib, json
files = {"model.bin": bytes(range(256)) * 40, "tokenizer.json": b'{"vocab": ["a", "b"]}',
"config.yaml": b"layers: 4\nheads: 8\n"}
manifest = {n: hashlib.sha256(b).hexdigest() for n, b in files.items()}
print(json.dumps({k: v[:16] for k, v in manifest.items()}, indent=1))
received = dict(files)
tampered = bytearray(received["model.bin"]); tampered[5000] ^= 1
received["model.bin"] = bytes(tampered)
received["config.yaml"] = b"layers: 4\nheads: 8\ntrust_remote_code: true\n"
for n, b in sorted(received.items()):
ok = hashlib.sha256(b).hexdigest() == manifest[n]
print(f"{n:15s} {'OK' if ok else 'MISMATCH'}")
Expected output:
{
"model.bin": "e96760a87768717b",
"tokenizer.json": "f63d2ddabc61f2d9",
"config.yaml": "e5bbb570ca0fcb87"
}
config.yaml MISMATCH
model.bin MISMATCH
tokenizer.json OK
In Lab H, a single flipped bit in model.bin and an added line in config.yaml both change their hashes, while the untouched tokenizer.json still matches. The manifest must come from a trusted source, such as the vendor's signed release, or an attacker who changes a file could change the manifest too.
8.4 Adversarial tests
Add tests that try to break your code: oversized inputs, wrong types, injection-like strings, requests for other users' data. Keep these in the test suite permanently.
For the booking parser, adversarial tests include a date written in words, a patient ID belonging to someone else, a clinic code with trailing spaces, and model output containing two bookings. Each test records the expected rejection. Once added, these tests protect against regressions: a later change that accidentally relaxes a check fails at once, before it reaches patients.
8.5 Worked example (synthetic LCH case)
DocSort's vendor sent an updated model. Rina's integrity check compared the hash with the vendor's signed manifest and found a mismatch for the configuration file. Investigation showed a contractor had edited the file to enable a debug mode that logged full letter text. The change was reverted, and LCH required all model changes to come through the signed release process.
The debug mode matches Lab H's tampered configuration line in spirit: a small text change with a large effect on privacy.
8.6 Errors and limits
- Policy logic mixed into prompt text.
- Loading model files without integrity checks.
- Tests with only benign inputs.
Policy written into the prompt can be argued away by a clever message, while policy in code cannot. Loading model files without checking hashes would have let the contractor's debug mode reach production at LCH. Tests with only benign inputs pass even when the parser accepts dangerous output; the hostile test file for the booking parser exists precisely to prevent that, and it grows with every incident.
8.6b Tests that guard the parser
The module that turns model output into a booking request is the most security-relevant code in Ask Lumen, because a malformed or manipulated answer could cancel the wrong appointment. LCH keeps a test file of hostile outputs — extra fields, wrong dates, two appointment IDs — and every change to the parser must pass it before merging.
Each hostile output in the file comes from a real failure or a review finding, so the file records the parser's history of attacks as well as protecting against them. When a new kind of failure appears, a test for it is added before the fix.
Exercises
- EX022 — Lab reading. Which files fail the hash check in the lab and why? Rubric: 3 = model.bin and config.yaml, both changed; 2 = one; 1 = vague; 0 = wrong.
- EX023 — JSON validation. List four checks for model-produced booking JSON. Rubric: 3 = valid JSON, allowed fields only, types, ranges/ownership; 2 = three; 1 = one or two; 0 = none.
- EX024 — Debug mode. Why was the debug-mode change a security issue? Rubric: 3 = logged sensitive health data, expanding exposure; unauthorised change; 2 = partial; 1 = vague; 0 = wrong.
Synthesis
Structured programs, strict parsing, verified dependencies and adversarial tests make AI applications harder to break and easier to trust.
Continue studying — chapter 08 of 40
← Chapter 07 (draft) · Course chapter index · Chapter 09 (draft) →