Chapter 4 — Digital forensics and malware analysis: competing explanations
Lesson map
Lesson 04 published. Map, example, EX021.
flowchart TD
N0["Learning outcomes"]
N1["4.1 Why alternatives matter"]
N2["4.2 Observation"]
N3["4.3 Tests"]
N4["4.4 Malware alternatives"]
N5["4.5 Confirmation bias"]
N6["4.6 Recording reasoning"]
N7["4.7 Implication for the study"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX021"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["4.1 Why alternatives matter"]
N2["4.2 Observation"]
N3["4.3 Tests"]
N4["4.4 Malware alternatives"]
N5["4.5 Confirmation bias"]
N6["4.6 Recording reasoning"]
N7["4.7 Implication for the study"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX021"]
[Learning outcomes]
|
v
[4.1 Why alternatives matter]
|
v
[4.2 Observation]
|
v
[4.3 Tests]
|
v
[4.4 Malware alternatives]
|
v
[4.5 Confirmation bias]
|
v
[4.6 Recording reasoning]
|
v
[4.7 Implication for the study]
|
v
[Worked example] --> [Exercise EX021]
Learning outcomes
You will (1) state competing explanations for a forensic observation; (2) design tests that separate them; (3) apply the idea to triage results; (4) avoid confirmation bias.
4.1 Why alternatives matter
Casey (2011) emphasises that examiners must consider alternative explanations before reaching conclusions. ISO/IEC 27042 likewise addresses the analysis and interpretation of digital evidence, including the need for documented, repeatable reasoning. For Tomás, this applies to his research as much as to casework: an unexpected result needs alternative explanations considered before it is interpreted. His notes list each explanation with the test that could rule it out, as he would for an examination in his police role. Dr Osei reviews these notes at each supervision meeting, starting with the three explanations for the low-reduction pilot image.
4.2 Observation
In a pilot run, reduction on one image was only 40%, far below the others. Tomás lists explanations: (a) the image had more user files; (b) the OS build was newer than the reference set; (c) the hashing tool failed on some files. A result this far from the others could reflect a real property of the image, or a fault in his method. Listing the explanations before testing any of them keeps him from settling on the first that seems plausible. Each explanation is written in the pilot notes with the test he plans to use.
4.3 Tests
(a) Count user files: similar to others. (b) Compare OS build date with reference set release: the build was newer, so many system files lack entries. (c) Rehash with a second tool: identical hashes. Explanation (b) survives. Each test was chosen to rule out one explanation independently of the others. The surviving explanation changes how the study must be designed, because reduction now appears to depend on how current the reference set is relative to the operating system. The pilot notes record every test and result, including the two rejected explanations, so a reviewer can follow the reasoning.
4.4 Malware alternatives
When an unknown executable remains after filtering, explanations include malware, a custom in-house tool, or a software update newer than the reference set. Static examination of the synthetic sample's metadata — signer, version resource, install path — helps separate them without running it. These explanations matter because the synthetic images include software updates that postdate the reference set, so an unknown executable is not automatically suspicious. Tomás records what each metadata feature showed and which explanation it supports. He does not run unknown samples, in keeping with the course's ethical boundary.
4.5 Confirmation bias
Examiners who expect malware may read every unknown file as hostile. Blind review, where a colleague assesses without the hypothesis, reduces this risk. In his study, designed misses are known to Tomás but not to the examiners in the agreement study, who classify events without knowing which files were planted. He also asks a colleague to review a sample of his own classifications without telling them his expectations, and records where they differed and why. Blind review is also used for the hand checks of unknown executables.
4.6 Recording reasoning
Each explanation, its test and its result go in the notes, so a reviewer can see why rejected options were rejected. Recording rejected explanations protects the work in two ways. A reviewer can check whether the tests were adequate, and a later researcher can see which explanations were already considered. Tomás's pilot notes for the low-reduction image are written this way: three explanations, three tests, and a short statement of which explanation survived and why. The same format is used for unknown executables in the main run.
4.7 Implication for the study
Reduction depends on reference-set currency; Tomás adds "days between OS build and reference release" as a variable. The pilot finding turns into a design change: rather than treating the low-reduction image as an outlier, Tomás measures the gap between OS build and reference release for every image. The main run can then show how reduction changes with that gap, which is more useful to practitioners than a single average. He records the change and its reason in the protocol before the main run begins.
Worked example
The pilot notes read: "Image L3: 40% reduction. H(a) user volume — rejected, 102 user files vs mean 98. H(b) newer OS — supported, build 73 days after reference release. H(c) tool fault — rejected, second tool matched all hashes."
Synthetic case. Image L3 and its numbers are invented.
Errors and limits
Surviving one test does not prove an explanation; it is the best-supported so far.
Exercises
- EX021 — Three explanations. List the three explanations for low reduction on L3. Rubric: 3 = user volume, newer OS, tool fault; 2 = two; 1 = one; 0 = none.
- EX022 — Tool test. How was tool failure tested? Rubric: 3 = rehash with second tool, identical; 2 = partial; 1 = wrong; 0 = none.
- EX023 — Unknown executable. Name three explanations for an unknown executable after filtering. Rubric: 3 = malware, in-house tool, newer update; 2 = two; 1 = malware only; 0 = none.
- EX024 — Static separation. Which metadata help separate them without execution? Rubric: 3 = signer, version resource, path; 2 = two; 1 = run it; 0 = none.
- EX025 — Bias control. Describe one control for confirmation bias. Rubric: 3 = blind review by colleague; 2 = vague; 1 = minimal attempt; 0 = none.
- EX026 — New variable. What variable did the study add, and why? Rubric: 3 = days between build and reference release; currency drives reduction; 2 = partial; 1 = wrong; 0 = none.
Synthesis
Name rival explanations, test each, record why rejected ones failed, and let the result reshape the study.
Continue studying — chapter 04 of 08
← Chapter 03 (draft) · Course chapter index · Chapter 05 (draft) →