Chapter 14 — Software engineering | Engineering Digital forensics and malware analysis

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 14 — Software engineering

Lesson map

Lesson 14 published. Map, example, EX040.

flowchart TD
  N0["Learning outcomes"]
  N1["14.1 Lifecycle"]
  N2["14.2 Requirements"]
  N3["14.3 Testing pyramid"]
  N4["14.4 Change management"]
  N5["14.5 Defects"]
  N6["14.6 Documentation"]
  N7["14.7 Reproducible builds"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX040"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["14.1 Lifecycle"]
  N2["14.2 Requirements"]
  N3["14.3 Testing pyramid"]
  N4["14.4 Change management"]
  N5["14.5 Defects"]
  N6["14.6 Documentation"]
  N7["14.7 Reproducible builds"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX040"]

[Learning outcomes]
    |
    v
[14.1 Lifecycle]
    |
    v
[14.2 Requirements]
    |
    v
[14.3 Testing pyramid]
    |
    v
[14.4 Change management]
    |
    v
[14.5 Defects]
    |
    v
[14.6 Documentation]
    |
    v
[14.7 Reproducible builds]
    |
    v
[Worked example] --> [Exercise EX040]

Learning outcomes

You will (1) apply a development lifecycle to forensic tools; (2) write requirements and tests; (3) manage change; (4) support tool validation.

14.1 Lifecycle

Requirements, design, implementation, testing, release, maintenance. For forensic tools, each stage leaves records a validator can inspect.

For EvidenceDesk, the records are a requirements list, design notes, the pull request history, test results for each release and a log of reported defects. A validator assessing whether the tool suits casework reads these rather than relying on the developer's word. Gaps in the records count as risks: if nobody can show what was tested before a release, the release is treated as untested.

14.2 Requirements

"The tool shall compute SHA-256 of each input and report mismatch against a stored value" is testable. "The tool shall be fast" is not until quantified.

Quantifying speed might read: 'The tool shall verify a 500 GB image at no less than 300 MB per second on the reference workstation.' Each requirement gets an identifier, so tests and defects can refer to it. Requirements also cover behaviour on failure, for example 'shall exit with a non-zero status and log the reason if any input cannot be read', because failure modes are where forensic tools most often mislead.

14.3 Testing pyramid

Many unit tests, fewer integration tests, a few end-to-end tests on synthetic evidence sets. NIST's Computer Forensics Tool Testing programme shows how test cases tie to specific requirements.

Unit tests are fast and pin down small behaviours, such as converting one timestamp format correctly. Integration tests check that components work together, such as that hashing and logging agree on file names. End-to-end tests run the whole tool on a prepared evidence set and compare outputs with expected results. Each test names the requirement it covers, so gaps in coverage are visible.

14.4 Change management

Each change is reviewed and linked to a requirement or defect. Releases carry version numbers and release notes listing changes that might affect results.

Release notes separate cosmetic changes from those that could alter results, such as a new parser for a file-system feature. Examiners read the second group before upgrading, and the lab may rerun its validation set before allowing the new version on casework. A change with no linked requirement or defect gets questioned in review, because unexplained changes are where silent regressions hide.

14.5 Defects

Defects are logged with severity. A defect that could change findings triggers review of past cases run with the affected version.

Severity depends on what the defect can do to findings. A typo in the help text is low. A parser that skips some entries in a log is high, because it could hide evidence. When a high-severity defect is confirmed, the lab lists every case processed with the affected versions, assesses whether the defect could have changed results, and records the outcome of that review in each case file.

14.6 Documentation

User guide, known limitations and validation summary ship with each release.

Known limitations are stated plainly, for example 'does not parse compressed NTFS attributes; affected files are listed but their content is not hashed'. The validation summary lists the test set, the version tested, the pass and fail results and the date. An examiner who uses the tool outside its validated scope has to justify that decision in their own notes. The user guide includes worked examples with expected output, so an examiner can confirm that a fresh installation behaves the way the validation describes before using it on a case.

14.7 Reproducible builds

Pinning versions and recording the build environment lets anyone rebuild the same binary from source.

Pinning means recording exact versions of the compiler or interpreter, every dependency and the build options. The build environment can be captured as a container image or a detailed manifest. Two people building the same source at the same commit should get byte-identical outputs, so a binary used on a case can be checked against its source years later by hashing a fresh build. The release record stores the hash of the binary.

Static analysis and review

Automated checks catch common faults such as unhandled exceptions or unused variables before human review. Code review then focuses on logic: does this function do what the requirement says, and is every input validated? Priya keeps review comments with the change, so an auditor can see what was questioned and how it was resolved. Over a semester, her team finds that review caught more result-affecting faults than testing alone, and she records that observation, with counts, in the project log.

Worked example

A defect report: "Timeline export drops events with identical timestamps." Severity: high (could omit evidence). Priya fixes it, adds a test with duplicate timestamps, releases 1.2.1 and lists affected versions so Northfield can review cases.

Synthetic case. The defect is invented.

Errors and limits

Good process lowers risk; it does not prove a tool is correct for untested inputs.

Exercises

EX040 — Testable requirement. Rewrite "the tool shall be fast" testably. Rubric: 3 = quantified, e.g. hash 100 GB in under 20 min on stated hardware; 2 = partial; 1 = unchanged; 0 = none.
EX041 — High-severity defect. Why was the duplicate-timestamp defect high severity? Rubric: 3 = could omit evidence from findings; 2 = partial; 1 = cosmetic; 0 = none.
EX042 — Release notes. What must release notes include for forensic users? Rubric: 3 = changes affecting results, affected versions; 2 = partial; 1 = marketing; 0 = none.

Synthesis

Disciplined software engineering gives validators the evidence they need and tells labs when past results need review.

Continue studying — chapter 14 of 50

← Chapter 13 (draft) · Course chapter index · Chapter 15 (draft) →