Chapter 3 — Programming foundations | Technologist Digital forensics and malware analysis

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 03 published. Map, example, EX007.

flowchart TD
  N0["Learning outcomes"]
  N1["3.1 Why technologists script"]
  N2["3.2 Read-only by default"]
  N3["3.3 Clear output"]
  N4["3.4 Standard library first"]
  N5["3.5 Version control"]
  N6["3.6 Testing"]
  N7["3.7 Comments"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX007"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["3.1 Why technologists script"]
  N2["3.2 Read-only by default"]
  N3["3.3 Clear output"]
  N4["3.4 Standard library first"]
  N5["3.5 Version control"]
  N6["3.6 Testing"]
  N7["3.7 Comments"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX007"]

[Learning outcomes]
    |
    v
[3.1 Why technologists script]
    |
    v
[3.2 Read-only by default]
    |
    v
[3.3 Clear output]
    |
    v
[3.4 Standard library first]
    |
    v
[3.5 Version control]
    |
    v
[3.6 Testing]
    |
    v
[3.7 Comments]
    |
    v
[Worked example] --> [Exercise EX007]

Learning outcomes

You will (1) write small Python scripts safely; (2) read files without modifying them; (3) print clear, checkable output; (4) use version control for scripts used in investigations.

3.1 Why technologists script

Investigations repeat tasks: hash many files, filter logs, compare lists. Scripts make the work faster and repeatable, but only if they are correct and recorded.

Repeatability matters most when someone challenges a result. If Jonah hashes two hundred files by hand and makes one typing error, nobody can tell which digest is wrong. A script run twice on the same input gives the same output, and a reviewer can read exactly what it did. The cost is that a wrong script is wrong every time, so testing comes first.

3.2 Read-only by default

Open evidence files in binary read mode ("rb"). Write outputs to a separate folder. Never "fix" an evidence file.

Binary mode also avoids newline translation on some systems, which would alter the bytes a hash function sees. Jonah keeps evidence in one folder, outputs in another, and never points a script's output path at the evidence folder. If a script must create temporary files, they go in the output folder too, so the evidence copy stays exactly as it was received.

3.3 Clear output

Print what was processed, how many items, and any errors. A silent script that prints nothing on failure is dangerous.

Jonah's improved listing script shows the form: it reports how many files it listed, how many it could not open, and the path of each failure. A reader can then check the totals against the folder and see immediately whether anything was missed. A script that reports success without counts gives no way to notice that half the folder was skipped.

3.4 Standard library first

Python's standard library covers hashing (hashlib), CSV (csv), JSON (json), dates (datetime), regular expressions (re) and file handling (os, pathlib). Using it avoids installing extra packages on sensitive machines.

On a city workstation or an evidence machine, installing packages means downloading code from outside, which changes the system and adds software whose behaviour has not been reviewed. Jonah's hashing function uses only hashlib and reads files in chunks, so it runs on any machine with Python 3 and gives results another analyst can reproduce without a matching set of packages.

3.5 Version control

Scripts live in a repository; each change has a message. Reports cite the version used.

For Jonah this means each script used in a case is committed before it is run, and the commit identifier is written in the case notes next to the output. If the script later changes, anyone can return to the exact version that produced a finding. A script copied around by email or edited in place leaves no record of what actually ran. Renata also asks for a short message on each commit that says why the change was made.

3.6 Testing

Run scripts on synthetic inputs with known answers before using them on case data.

Jonah's hashing function was checked against a second tool on five synthetic files and an empty file, and the files and expected digests were kept in the repository. The empty file is a deliberate edge case: it catches mistakes in how a function handles reading zero bytes. Tests are rerun after every change, so a later edit cannot quietly break a result. A test that passes on synthetic data shows the script works as intended; it does not show the evidence is complete.

3.7 Comments

A short comment explains why, not what: "binary mode avoids newline translation".

Good comments record decisions a reader might question: why a file is read in chunks, why a failure is logged rather than raised, why a field is treated as UTC. Renata reviews Jonah's scripts by reading the comments first; where a comment is missing, she asks the question it should have answered, and the answer becomes the comment. Comments that merely restate the code add nothing and drift out of date when the code changes.

Hashing a file

A short function opens a file in binary mode, reads it in chunks into hashlib.sha256(), and returns the hex digest. Jonah tests it against a second tool on five synthetic files and on an empty file, confirming identical results before using it. He keeps the test files and expected digests in the repository so anyone can rerun the check after changes. When a classmate later edits the function, the stored test immediately shows whether results still match.

Worked example

Jonah's first script lists files in a folder with their sizes. Renata asks what happens if a file cannot be opened. It crashes. He adds error handling that records "permission denied: path" and continues, then prints totals: 211 listed, 1 error.

Synthetic case. The folder is invented.

Errors and limits

A script reflects its author's assumptions; peer review catches assumptions the author cannot see.

Exercises

EX007 — Read mode. Which mode should evidence files be opened in, and why?
Show marking criteria
Rubric: 3 = binary read-only "rb"; no changes, no translation; 2 = partial; 1 = write; 0 = none.
EX008 — Silent failure. Why is a silent script dangerous?
Show marking criteria
Rubric: 3 = failures go unnoticed; incomplete results trusted; 2 = partial; 1 = fine; 0 = none.
EX009 — Error handling. What did Jonah's improved script do on an unreadable file?
Show marking criteria
Rubric: 3 = logged error, continued, printed totals 211/1; 2 = partial; 1 = crashed; 0 = none.

Synthesis

Read-only, verbose, tested and versioned scripts are safe investigative tools.

Continue studying — chapter 03 of 30

← Chapter 02 (draft) · Course chapter index · Chapter 04 (draft) →