◇ OpenCyber Academy
Technical · Chapter 4 · English

Chapter 4 — Scripting and data formats

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 4 — Scripting and data formats

Lesson map

Lesson 04 published. Map, example, EX013.

flowchart TD
  N0["Learning outcomes"]
  N1["4.1 Formats"]
  N2["4.2 Reading with Python"]
  N3["4.3 Small checks"]
  N4["4.4 Never print secrets"]
  N5["4.5 Synthetic data"]
  N6["A worked check"]
  N7["Handling errors"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX013"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["4.1 Formats"]
  N2["4.2 Reading with Python"]
  N3["4.3 Small checks"]
  N4["4.4 Never print secrets"]
  N5["4.5 Synthetic data"]
  N6["A worked check"]
  N7["Handling errors"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX013"]

[Learning outcomes]
    |
    v
[4.1 Formats]
    |
    v
[4.2 Reading with Python]
    |
    v
[4.3 Small checks]
    |
    v
[4.4 Never print secrets]
    |
    v
[4.5 Synthetic data]
    |
    v
[A worked check]
    |
    v
[Handling errors]
    |
    v
[Worked example] --> [Exercise EX013]

Learning outcomes

You will (1) read JSON, YAML and environment files; (2) write small Python checks; (3) avoid printing secrets.

4.1 Formats

Cloud settings arrive as JSON, Kubernetes manifests as YAML, and application settings as environment files with KEY=value lines. Tomás learns to recognise each. In Fairway's work these are concrete: the bucket settings export is JSON, the tracker's deployment file is YAML, and app.env and worker.env hold KEY=value lines. Each of these formats has its own risks for a script that reads it. JSON is strict, YAML allows many forms that simple readers miss, and environment files often hold credentials, which is why the secrets lab later reads them carefully.

4.2 Reading with Python

Python's standard library reads JSON directly, and simple line-based formats with string methods. The labs in this course use these, with no extra packages. Keeping to the standard library means the labs run on the college lab machines and Fairway's admin laptop without installing anything. It also means no unreviewed third-party package ever touches the exported settings. For YAML, Tomás reads only the simple key-value lines his checks need, and he notes that anything more complex would need a proper parser and a review.

4.3 Small checks

A check loads records, compares each with an expected value and prints differences. Counts come first, then details. Counts come first because a reader can see at once whether a run found nothing, a few items or something alarming. The verification lab follows this pattern, ending with '6 differences across 4 buckets' after listing each bucket. Tomás also prints which expected value each record was compared with, so a reviewer can check the policy and the result together. A check that prints nothing at all is treated as a failure, not a pass.

4.4 Never print secrets

When a check looks for secrets, it reports the key name and file, never the value. Printed secrets end up in logs and screenshots. The secrets lab follows this rule: it prints DB_PASSWORD, API_TOKEN and CLOUD_SECRET_KEY by name and file only. A value printed once can end up in the pipeline logs, the team channel and a screenshot in a report, each kept longer than the secret's own lifetime. Grace checks every script's output format before it runs against real configuration.

4.5 Synthetic data

All labs use synthetic records; values that look like secrets are written as . Synthetic data lets Tomás practise without opening real manifests, which contain customer addresses he is not authorised to read for analysis. The labs keep realistic structure, such as bucket names and event types, while names, times and values are invented. That lets anyone at Hollins rerun the labs, and none of their outputs can leak real customer or credential data. Grace checks new lab data for anything that resembles a real record before it is shared.

A worked check

Tomás's first check has four parts: load the export, define the expected value, compare each record and print counts then details. He keeps the expected values at the top of the script so that anyone reviewing can see the policy without reading the loop. When the policy changes, only that block changes.

He tests the check on a small synthetic file where he knows the answer, including one bucket that should pass and one that should fail. Only after both behave as expected does he run it on the full synthetic export. Testing with known answers is a habit he keeps for every lab in the course.

Handling errors

If a record is missing a field, the check reports it rather than crashing or silently skipping it, because a missing setting is itself a finding.

Worked example

Tomás writes a check that reads a JSON export of bucket settings and prints buckets without encryption. Grace reviews it and asks him to add a count at the top and a comment explaining the expected value.

Synthetic case. Data is synthetic.

Errors and limits

Simple parsers may mis-read complex YAML; use them only on simple synthetic examples.

Exercises

Synthesis

Read common formats safely and print results, not secrets.

Continue studying — chapter 04 of 18

← Chapter 03 (draft) · Course chapter index · Chapter 05 (draft) →