◇ OpenCyber Academy
Master's · Chapter 2 · English

Chapter 2 — Digital forensics and malware analysis: advanced seminar

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 2 — Digital forensics and malware analysis: advanced seminar

Lesson map

Lesson 02 published. Map, example, EX008.

flowchart TD
  N0["Learning outcomes"]
  N1["2.1 Process models"]
  N2["2.2 How known-file filtering works"]
  N3["2.3 Known-good versus known-bad"]
  N4["2.4 Malware connection"]
  N5["2.5 Seminar critique"]
  N6["2.6 Fuzzy hashing"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX008"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["2.1 Process models"]
  N2["2.2 How known-file filtering works"]
  N3["2.3 Known-good versus known-bad"]
  N4["2.4 Malware connection"]
  N5["2.5 Seminar critique"]
  N6["2.6 Fuzzy hashing"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX008"]

[Learning outcomes]
    |
    v
[2.1 Process models]
    |
    v
[2.2 How known-file filtering works]
    |
    v
[2.3 Known-good versus known-bad]
    |
    v
[2.4 Malware connection]
    |
    v
[2.5 Seminar critique]
    |
    v
[2.6 Fuzzy hashing]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX008]

Learning outcomes

You will (1) explain the forensic process models used in research; (2) relate hash-based triage to them; (3) critique a published method in seminar; (4) connect malware analysis to known-file filtering.

2.1 Process models

In NIST SP 800-86 terms, filtering falls in the second of four phases: collection, examination, analysis and reporting. ISO/IEC 27043 sets out investigation principles and processes across readiness, initialisation, acquisition and investigation. Tomás places filtering in examination: it reduces data before analysis. Placing filtering in examination matters for his argument: decisions made there shape what an examiner later analyses and reports, so a file set aside at this stage may never be looked at again. Tomás uses both models to explain where errors introduced by filtering could enter an investigation.

2.2 How known-file filtering works

Each file on the image is hashed; hashes found in a reference set of known software (such as the NSRL) are set aside as "known". The NSRL documentation describes it as a collection of hashes of published software; it does not label files good or bad. The distinction between known and good is the centre of his study: a file set aside as known is identical to published software, but nothing in the match says whether it was used for a legitimate purpose. Examiners sometimes treat known files as harmless, which is the assumption he tests.

2.3 Known-good versus known-bad

Some labs maintain "known-bad" sets of contraband or malware hashes. Matching a known-bad hash flags a file for attention; matching a known software hash only shows the file equals a published file. The two kinds of set work in opposite directions: a known-bad match draws attention to a file, while a known software match removes it from view. Tomás studies only the second, because that is where relevant files could disappear without anyone noticing. His thesis explains the difference early, so readers do not confuse the two, and his synthetic images use only a known software reference set.

2.4 Malware connection

Malware may replace a system file. The replaced file's hash no longer matches the reference set, so filtering correctly leaves it for review. But if an attacker uses a legitimate, unmodified tool, its hash matches and filtering hides it. Sikorski and Honig (2012) stress that legitimate binaries can be misused. The second case is the one his study is designed to measure. A remote-administration tool used by an intruder is identical to the published version, so filtering sets it aside as known. Tomás plants such cases in his synthetic images as designed misses, so he can count how often filtering hides them.

2.5 Seminar critique

Tomás presents a paper reporting 70% reduction. Peers ask: what images? Which version of the reference set? Were relevant files excluded? The paper does not say, which limits its weight. The seminar shapes Tomás's own reporting: he decides to state the images, the reference set version and the number of relevant files set aside for every result. That way his figures can be judged in a way the paper's 70% could not. Dr Osei asks him to keep the seminar's three questions as a checklist for his own results chapter.

2.6 Fuzzy hashing

Similarity digests (Roussev's sdhash, Kornblum's ssdeep) find near-matches; Tomás keeps them out of scope to bound his study. Similarity digests could catch modified versions of known tools, which is a related but different problem from his. Including them would add new measures, settings and error types, enough for a separate study. Tomás describes them in the literature review so readers know why they are excluded, and he names them as future work in the thesis. Keeping them out lets him answer one question well within a master's timetable.

Worked example

On a synthetic image, a remote-administration tool appears in the reference set because it is published software. Filtering marks it "known". In the synthetic scenario it was used by an intruder. Tomás records this as a category of miss: legitimate tool, illegitimate use.

Synthetic case. The image and intrusion are invented.

Errors and limits

"Known" is not "irrelevant"; reports must not imply that filtered files were examined and cleared.

Exercises

Synthesis

Filtering is an examination-phase data reduction step; its matches mean "published", never "innocent".

Continue studying — chapter 02 of 08

← Chapter 01 (draft) · Course chapter index · Chapter 03 (draft) →