Chapter 13 — Digital forensics and malware analysis: troubleshooting
Lesson map
Lesson 13 published. Map, example, EX049.
flowchart TD
N0["Learning outcomes"]
N1["13.1 A method"]
N2["13.2 Case problem: missing events"]
N3["13.3 Working through it"]
N4["13.4 Resolution"]
N5["13.5 Tool faults versus evidence"]
N6["13.6 Escalation"]
N7["13.7 Recording"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX049"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["13.1 A method"]
N2["13.2 Case problem: missing events"]
N3["13.3 Working through it"]
N4["13.4 Resolution"]
N5["13.5 Tool faults versus evidence"]
N6["13.6 Escalation"]
N7["13.7 Recording"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX049"]
[Learning outcomes]
|
v
[13.1 A method]
|
v
[13.2 Case problem: missing events]
|
v
[13.3 Working through it]
|
v
[13.4 Resolution]
|
v
[13.5 Tool faults versus evidence]
|
v
[13.6 Escalation]
|
v
[13.7 Recording]
|
v
[Worked example] --> [Exercise EX049]
Learning outcomes
You will (1) troubleshoot collection and analysis problems methodically; (2) separate tool faults from evidence questions; (3) escalate when stuck; (4) record troubleshooting.
13.1 A method
Describe the symptom precisely, check the simplest causes first, change one thing at a time, and record each step and its result. This avoids random changes that hide the cause.
Precise description means stating the host, time range and what is missing, such as 'no EDR events on PHARM-PC3 between 11:20 and 11:40'. Checking simple causes first saves time, because a wrong filter is far more common than an attacker deleting logs. Each step and its result goes into the case log as it happens.
13.2 Case problem: missing events
PHARM-PC3's EDR timeline shows no events between 11:20 and 11:40, although the PC was in use. Possible causes: agent stopped, network loss, console filter, or attacker tampering.
The gap began five minutes after the stick was mounted on PHARM-PC3, lasted twenty minutes and spanned the containment of PHARM-PC3 at 11:31. If the tool had removed events, that would change the case badly. Sana lists every possible cause before checking any of them. Each candidate cause is written as a question with the record that would answer it.
13.3 Working through it
Sana checks the console filter first — none applied. The agent health log shows the agent was running. The switch log shows the pharmacy switch port flapping between 11:19 and 11:41. Events were buffered locally and arrived later, stamped with their original times but filed under a later upload batch.
Each check rules out one cause: the filter, then the agent, then the network path. The switch log is owned by the network team, so Sana requests it through their queue with the case reference. The flapping port explains why events reached the console late but kept their original event times.
13.4 Resolution
A query by event time rather than upload time returns the missing events. No tampering is indicated.
Thirty-eight events were recovered by querying on event time instead of upload time. Sana reruns her earlier PHARM-PC3 queries with the corrected field, so no other result depends on the faulty query. The recovered events are added to the timeline with a note on how they were found. The faulty query is kept in the case log beside the corrected one, so a reviewer can see exactly what changed between them.
13.5 Tool faults versus evidence
A tool fault (wrong query field) looked like an evidence gap. Before reporting "logs deleted", operators rule out tool and query faults.
Reporting 'logs deleted' would have suggested tampering and changed the investigation's direction and urgency. The real cause was a query that used the upload time field, which is a tool and method problem rather than evidence of an attacker. Sana adds a line to the runbook to query by event time. The runbook change is reviewed and tested like any other document edit.
13.6 Escalation
If two hours of methodical work do not explain a problem, Sana escalates to Owen with her notes, rather than guessing.
Two hours is a limit agreed with Owen, so escalating is part of the method rather than an admission of failure. Her notes travel with the escalation, so Owen does not repeat the checks she has already done. In this case the gap was explained within the limit, and no escalation was needed. Escalation notes list each check, its result and the records consulted.
13.7 Recording
The troubleshooting steps go into the case log, because a gap explained wrongly can mislead the whole investigation.
The log records each check, its result and the conclusion: delivery delay, not tampering. A later reader who sees a twenty-minute gap in PHARM-PC3's timeline can find the explanation instead of reopening the question. The note also names the switch log export that supports the conclusion. The explanation is cross-referenced from the PHARM-PC3 timeline entry for 11:20. The note gives the time each check was made and who made it.
Common symptoms
Other symptoms operators meet include a capture tool that refuses to start because of low disk space on the kit drive, a triage archive that will not open because the transfer was interrupted, and timestamps that appear in two different zones. Each has a simple first check: free space, a hash comparison with the manifest, and the export's time-zone setting.
Worked example
Sana's log: "11:20–11:40 gap on PHARM-PC3. Filter: none. Agent: running. Switch port flapped 11:19–11:41. Re-query by event time: 38 events recovered. Conclusion: delivery delay, not tampering."
Synthetic case. The logs are invented.
Errors and limits
Methodical troubleshooting still depends on available logs; some gaps cannot be explained.
Exercises
- EX049 — Method. Describe the troubleshooting method. Rubric: 3 = describe symptom, simple causes first, one change at a time, record; 2 = partial; 1 = random; 0 = none.
- EX050 — Gap cause. What caused the PHARM-PC3 gap? Rubric: 3 = switch port flapping; buffered events filed by upload time; 2 = partial; 1 = tampering; 0 = none.
- EX051 — Tool versus evidence. What lesson about tool faults does the case teach? Rubric: 3 = rule out query/tool faults before claiming deletion; 2 = partial; 1 = vague; 0 = none.
- EX052 — Escalation. When does Sana escalate? Rubric: 3 = after two hours unexplained, with notes; 2 = partial; 1 = never; 0 = none.
Synthesis
Troubleshoot methodically, rule out tool faults before drawing conclusions, and escalate with notes.
Continue studying — chapter 13 of 18
← Chapter 12 (draft) · Course chapter index · Chapter 14 (draft) →