Chapter 3 — Programming I | Bachelor's Digital forensics and malware analysis

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 03 published. Map, example, EX007.

flowchart TD
  N0["Learning outcomes"]
  N1["3.1 Extensions versus content"]
  N2["3.2 Why it matters"]
  N3["Lab — magic bytes"]
  N4["3.3 Reading the result"]
  N5["3.5 Code structure"]
  N6["3.6 Reading safely"]
  N7["3.7 Extending the script"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX007"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["3.1 Extensions versus content"]
  N2["3.2 Why it matters"]
  N3["Lab — magic bytes"]
  N4["3.3 Reading the result"]
  N5["3.5 Code structure"]
  N6["3.6 Reading safely"]
  N7["3.7 Extending the script"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX007"]

[Learning outcomes]
    |
    v
[3.1 Extensions versus content]
    |
    v
[3.2 Why it matters]
    |
    v
[Lab — magic bytes]
    |
    v
[3.3 Reading the result]
    |
    v
[3.5 Code structure]
    |
    v
[3.6 Reading safely]
    |
    v
[3.7 Extending the script]
    |
    v
[Worked example] --> [Exercise EX007]

Learning outcomes

You will (1) write simple Python with dictionaries and loops; (2) identify file types by header bytes; (3) detect extension mismatches; (4) understand why extensions are unreliable.

3.1 Extensions versus content

A file's extension is part of its name and can be anything. Its content often begins with a signature ("magic bytes") that identifies the format: PDF files start with %PDF, PNG files with \x89PNG, ZIP-based formats such as DOCX with PK\x03\x04, Windows executables with MZ.

Renaming invoice.exe to invoice.pdf changes nothing inside the file; the first two bytes are still MZ. Many operating systems choose an icon from the extension, so the file looks like a document to the user while being a program. Reading the first bytes is therefore the first check an examiner applies to any suspicious file.

3.2 Why it matters

Attackers sometimes disguise executables as documents. Examiners compare declared extension with actual header to flag mismatches for closer inspection.

A mismatch does not prove malice: a user might have renamed a file by mistake, or an application might store data with an unusual extension. It is a reason to look closer, recorded as a finding with the header bytes shown. For ws-114, a mismatched file in the victim's Downloads folder would fit the phishing hypothesis and would be hashed and analysed in an isolated environment.

Lab — magic bytes

"""Compare file extensions with leading magic bytes (synthetic files)."""
MAGIC = {b"%PDF": "pdf", b"\x89PNG": "png", b"PK\x03\x04": "zip-container", b"MZ": "windows-executable"}
files = {"report.pdf": b"%PDF-1.7 ...", "photo.png": b"\x89PNG\r\n...", "invoice.pdf": b"MZ\x90\x00...",
         "notes.docx": b"PK\x03\x04..."}
for name, head in files.items():
    kind = next((v for k, v in MAGIC.items() if head.startswith(k)), "unknown")
    ext = name.rsplit(".", 1)[1]
    ok = kind.startswith(ext) or (ext == "docx" and kind == "zip-container")
    print(f"{name:12} header={kind:18} {'ok' if ok else 'MISMATCH'}")

Run with python3 lab_magic.py. Verified output:

report.pdf   header=pdf                ok
photo.png    header=png                ok
invoice.pdf  header=windows-executable MISMATCH
notes.docx   header=zip-container      ok

3.3 Reading the result

report.pdf, photo.png and notes.docx have headers consistent with their extensions (DOCX is a ZIP container). invoice.pdf begins with MZ, the signature of a Windows executable: MISMATCH. That is a strong reason to examine it further — without opening or running it on an examination machine.

Note that the lab compares names with headers held in memory as byte strings; it does not open the files with any application. Only invoice.pdf is flagged, and the next step is to hash it and record that hash in the case notes before any further analysis, so later copies can be matched to the original.

3.4 Limits of signatures

Signatures identify format, not intent. Some formats lack a fixed signature; others share one (many office formats are ZIPs). Validated tools use larger signature sets and deeper parsing.

Because DOCX, XLSX and JAR files all start with PK, a header of PK says only that the file is a ZIP container; the examiner must look inside at the listed contents to tell them apart. A file with a correct header can still contain malicious macros or scripts. Some formats, such as plain text, have no signature at all. Leah's report therefore says what the header check shows and what it cannot show.

3.5 Code structure

The lab uses a dictionary from signatures to types and a loop over files. Leah adds a comment explaining each signature's source so others can check it.

The dictionary keeps data separate from logic: adding a new format means adding one entry, not changing the loop. The loop takes the first matching signature, so order matters when one signature is a prefix of another. The special case for docx shows a limitation: a better design would map each container type to the list of extensions it may legitimately carry. Leah notes this as an improvement for the next version.

3.6 Reading safely

The script reads only the first bytes as data; it never executes anything.

In the lab the header bytes are written into the script, so no real file is touched. A version that reads real files should open them in binary mode, read a fixed number of bytes, and close them, never passing them to a program that interprets their content. Running such checks on a copy, not the original evidence, also preserves the hash of the original and keeps it valid as evidence.

3.7 Extending the script

Leah adds a test with a file shorter than any signature, confirming the script reports "unknown" rather than crashing.

Edge cases like this are where scripts fail in real casework: an empty file, a file that cannot be read because of permissions, or a filename without an extension. In the lab, rsplit on a name with no dot returns a one-item list, so taking [1] raises an IndexError, which is worth a second test. Each test is kept with the script so a later change cannot reintroduce the bug.

Worked example

In the ws-114 case, Grace's team finds an email attachment named "invoice.pdf". Leah's script flags its MZ header. The finding is recorded as: "File named invoice.pdf has a Windows executable header; it was not opened." Analysis is passed to the isolated malware lab.

Synthetic case. The attachment is invented.

Errors and limits

A matching header does not prove a file is harmless; a PDF can itself contain malicious content.

Exercises

EX007 — Mismatch. Which lab file mismatched, and what does that suggest?
Show marking criteria
Rubric: 3 = invoice.pdf, MZ header, disguised executable; 2 = file only; 1 = wrong; 0 = none.
EX008 — DOCX header. Why does notes.docx start with PK?
Show marking criteria
Rubric: 3 = DOCX is a ZIP container; 2 = partial; 1 = corruption; 0 = none.
EX009 — Header limits. What does a correct header not prove?
Show marking criteria
Rubric: 3 = file is harmless; 2 = partial; 1 = proves safe; 0 = none.

Synthesis

Trust content over names; use header checks to triage, never to clear files.

Continue studying — chapter 03 of 40

← Chapter 02 (draft) · Course chapter index · Chapter 04 (draft) →