Chapter 9 — Programming II | Engineering Digital forensics and malware analysis

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 9 — Programming II

Lesson map

Lesson 09 published. Map, example, EX025.

flowchart TD
  N0["Learning outcomes"]
  N1["9.1 From scripts to programs"]
  N2["9.2 Error handling"]
  N3["9.3 Logging"]
  N4["9.4 Testing"]
  N5["9.5 Read-only discipline"]
  N6["9.6 Dependencies"]
  N7["9.7 Code review"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX025"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["9.1 From scripts to programs"]
  N2["9.2 Error handling"]
  N3["9.3 Logging"]
  N4["9.4 Testing"]
  N5["9.5 Read-only discipline"]
  N6["9.6 Dependencies"]
  N7["9.7 Code review"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX025"]

[Learning outcomes]
    |
    v
[9.1 From scripts to programs]
    |
    v
[9.2 Error handling]
    |
    v
[9.3 Logging]
    |
    v
[9.4 Testing]
    |
    v
[9.5 Read-only discipline]
    |
    v
[9.6 Dependencies]
    |
    v
[9.7 Code review]
    |
    v
[Worked example] --> [Exercise EX025]

Learning outcomes

You will (1) structure larger programs with functions and modules; (2) handle errors without losing data; (3) log what a program did; (4) write tests.

9.1 From scripts to programs

Priya's verification script grows into a tool that processes many evidence files. She splits it into functions: read_chunks, hash_file, compare, report.

Each function does one job and can be tested on its own. read_chunks yields bytes without knowing about hashing, and hash_file uses it without knowing where the bytes come from. compare takes two digests and returns a verdict. report formats results. When Hugo later asks for SHA-512 support, only hash_file changes, and the tests for the other three functions show nothing else broke.

9.2 Error handling

Unreadable files must not crash the run silently. Each error is caught, logged with the file name and reason, and the run continues. The final summary counts successes and failures.

Catching every possible exception hides bugs, so Priya catches only the expected ones, such as permission errors, missing files and read failures, and lets anything else stop the run loudly. The summary line reads, for example, '412 verified, 3 failed (see log)'. A run with failures exits with a non-zero status, so any automation that calls the tool cannot mistake a partial run for a complete one.

9.3 Logging

Every run writes a log: start time, tool version, inputs, outputs, errors. Logs support the documentation SP 800-86 calls for and let a reviewer reproduce the run.

Each log line carries a timestamp in UTC, a severity and a message. The first lines record the tool version, the Python version, the hash algorithm and the full command line, so a reviewer can rebuild the exact invocation. Logs are written to the output folder and are never overwritten: each run gets a new file named with its start time.

9.4 Testing

Unit tests check each function with known inputs; an integration test runs the whole tool on a small synthetic evidence set with known answers.

The synthetic evidence set contains five files with known digests, one deliberately corrupted copy and one unreadable file. The integration test asserts four matches, one mismatch and one logged error, and it checks the summary line word for word. If someone changes the output format, the test fails, which forces a deliberate decision about whether reports and downstream tools need updating too.

9.5 Read-only discipline

The tool opens evidence read-only and writes outputs to a separate folder. A test checks that input modification times do not change.

Opening a file read-only stops writes through that handle, but some systems update access times on read. The test records each input's metadata before and after the run and fails if anything changed. On the lab workstations evidence also sits on a read-only mount, so a bug in the tool cannot reach the originals even if the code tries.

9.6 Dependencies

Using only the standard library keeps the tool easy to validate and reproduce; each extra package must itself be tracked and tested.

Every third-party package brings its own versions, defects and update cycle, and each one has to be validated along with the tool. Priya's verifier needs nothing beyond hashlib, os, argparse and logging, all from the standard library. If a later feature needs a package, she pins its exact version and records its hash in a requirements file, so the same code can be installed again later.

9.7 Code review

Hugo reviews her pull request and asks for one change: print the hash algorithm name beside every digest.

The request looks small, but it matters for evidence. A digest without its algorithm name can be misread years later, especially when MD5 and SHA-256 values sit side by side in old reports. Hugo also checks that the tests changed along with the output and that the commit message explains why. The review comments are kept with the pull request as part of the tool's validation history.

Version control

Every change to the tool is committed with a message describing why. Releases are tagged, and reports cite the tag. If a bug is found later, the lab can identify exactly which cases used the affected version and decide whether results need review, which is a practical form of quality assurance.

Worked example

On a folder of 200 synthetic files, one is locked by permissions. The first version crashed after 120 files with no summary. The revised version logs "permission denied: case7/notes.txt", finishes the remaining 79, and reports 199 hashed, 1 failed.

Synthetic case. The folder and files are invented.

Errors and limits

Tests show the tool works on the cases tested; untested formats remain a risk.

Exercises

EX025 — Crash fix. How did the revised tool handle the locked file? Rubric: 3 = logged reason, continued, summary 199/1; 2 = partial; 1 = skipped silently; 0 = none.
EX026 — Read-only test. How can a test confirm inputs were not modified? Rubric: 3 = compare hashes/mtimes before and after; 2 = partial; 1 = trust; 0 = none.
EX027 — Algorithm label. Why print the algorithm name beside each digest? Rubric: 3 = digest meaningless without algorithm; avoids confusion; 2 = partial; 1 = style; 0 = none.

Synthesis

Structured, logged, tested, read-only code is what makes forensic tooling trustworthy.

Continue studying — chapter 09 of 50

← Chapter 08 (draft) · Course chapter index · Chapter 10 (draft) →