OpenCyber AcademyLesson map
Lesson 03 published. Map, example, EX011.
flowchart TD
N0["Learning outcomes and connection"]
N1["3.1 The specialist report as evidence"]
N2["3.2 Forensic reports"]
N3["3.3 Cloud audit reports"]
N4["3.4 Identity traces"]
N5["3.5 Expert limitation"]
N6["3.6 Reconciling disagreements"]
N7["3.7 Confidence statements"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX011"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["3.1 The specialist report as evidence"]
N2["3.2 Forensic reports"]
N3["3.3 Cloud audit reports"]
N4["3.4 Identity traces"]
N5["3.5 Expert limitation"]
N6["3.6 Reconciling disagreements"]
N7["3.7 Confidence statements"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX011"]
[Learning outcomes and connection]
|
v
[3.1 The specialist report as evidence]
|
v
[3.2 Forensic reports]
|
v
[3.3 Cloud audit reports]
|
v
[3.4 Identity traces]
|
v
[3.5 Expert limitation]
|
v
[3.6 Reconciling disagreements]
|
v
[3.7 Confidence statements]
|
v
[Worked example] --> [Exercise EX011]
Learning outcomes and connection
You will be able to (1) read forensic, cloud-audit and identity-trace reports critically; (2) reconcile conflicting specialist findings; (3) request clarifying evidence precisely; and (4) write a confidence statement that a non-specialist executive and a legal adviser can both use. Chapter 2 gave you the command structure; this chapter equips the commander or specialist lead to integrate expert evidence. Chapter 8 extends the method to cases where evidence conflicts deeply.
3.1 The specialist report as evidence
At this level, much evidence reaches you as reports written by others: internal forensic analysts, external retained firms, cloud providers, vendors. A report is a chain of claims built on artefacts, tools and interpretation. Reading it well means separating (a) the artefacts examined, (b) the methods and tools (with versions), (c) the observations, (d) the interpretations, and (e) the stated limitations — and noticing when one is missing.
Calder's ransomware case shows why: the external firm's report H and the internal cloud team's report C each reached a sound conclusion from different artefact sets. Neither had seen the other's logs.
3.2 Forensic reports
A host forensic report should state the acquisition method (live response, disk image, memory capture), hashes, the timeline of artefacts (process execution, file creation, registry or configuration changes), interpretation (for example "initial execution via a malicious document at 10:02") and limits (timestamp reliability, anti-forensics, missing logs). NIST SP 800-86 describes the integration of forensic techniques into response; standards such as ISO/IEC 27037 cover identification, collection, acquisition and preservation of digital evidence.
Report H in Calder's ransomware case had most of these elements: appliance logs with 30-day retention, a jump-host image, and an interpretation of the 01:12 VPN session. What it lacked was a statement that IdP logs had not been examined, which was the limit that mattered.
3.3 Cloud audit reports
Cloud evidence comes from provider logs whose completeness depends on configuration and licence (bachelor's Chapter 31 showed the effect of missing data-read logs). A cloud report should state which log types were available, their retention, any gaps, and the query logic used. Provider statements about their own systems are evidence too, but with limited verifiability.
Report C at Calder was a cloud report of this kind: it used IdP sign-ins and control-plane logs to trace a phished administrator and the access key created minutes later. Its conclusion was strong on what those logs showed, but it had not checked whether the key was used from Calder's VPN pool or on-premises ranges, which is exactly the question Calder later sent back to the cloud team.
3.4 Identity traces
Identity investigations follow principals across IdP sign-ins, token issuance, application sessions and privileged operations. Common pitfalls: assuming a sign-in location is the user's location; ignoring non-interactive token refreshes; confusing the account that authenticated with the service that acted on its behalf.
The Calder reconciliation turned on such a trace. The 01:12 VPN session looked like an exploit to the host-based firm, but it authenticated with valid credentials of an account whose password had been reset by the phished cloud administrator on 30 May. Following the principal across the IdP, the cloud control plane and the VPN connected two reports that each saw only one segment of the path.
3.5 Expert limitation
Experts are fallible and have incentives: an external firm may be cautious to limit liability; an internal team may be reluctant to find gaps in its own controls; a vendor may minimize its product's role. Ask each expert: What did you examine? What did you not examine? What would change your conclusion? How confident are you, and why?
At Calder these questions produced the two clarifying requests: the external firm was asked to review VPN authentication logs for 28 May to 3 June for a named list of accounts, and the internal team was asked whether the 28 May access key was used from the VPN pool or on-premises ranges.
3.6 Reconciling disagreements
When reports disagree, tabulate: claim, report A's position and basis, report B's position and basis, what explains the difference (different artefacts, different time windows, different assumptions), and what evidence would discriminate. Often the disagreement dissolves into "they examined different things".
Calder's table is a model. On initial access, report H said VPN exploit on 3 June and report C said phished admin on 28 May. The difference was explained by artefact sets: H never saw IdP logs, and C never examined the VPN. The discriminating evidence was whether the 01:12 session used credentials of an account compromised on 28 May. Once checked, both reports turned out to be partly right.
3.7 Confidence statements
A confidence statement gives the conclusion, the confidence level on a defined scale, the main supporting evidence, the main alternative and why it is less likely, and what would change the assessment. Analytic standards in intelligence (for example the US Intelligence Community Directive 203) require such expressions of confidence and the distinction between assumptions and judgements; the same discipline is useful in incident reports.
Calder's statement follows this form: moderate confidence that initial access occurred on 28 May through phishing of a cloud administrator, with the IdP sign-ins, the access key and the 30 May reset as support, and the VPN-exploit theory as the main alternative.
3.8 Worked example (synthetic): initial access disputed
Fictional. Calder Group's Logistics division was hit by ransomware. Two reports disagree on initial access.
Report H (external forensic firm, host-based): initial access via exploitation of the internet-facing VPN appliance on 3 June, based on appliance logs showing an unusual session at 01:12 and subsequent RDP from the VPN pool to a jump host. Artefacts: appliance logs (30-day retention), jump-host image.
Report C (internal cloud team): initial access via a phished cloud administrator on 28 May, based on IdP sign-ins from an anonymizing service and creation of an access key used later to download a backup. Artefacts: IdP logs, cloud control-plane logs.
Reconciliation table: | Claim | H | C | Explanation | Discriminating evidence | |---|---|---|---|---| | Initial access | VPN exploit 3 June | Phished admin 28 May | Different artefact sets; H never saw IdP logs; C never examined the VPN | Whether the VPN session at 01:12 used credentials of an account compromised on 28 May; whether the cloud key's use links to on-premises hosts | | Earliest activity | 3 June | 28 May | Window: VPN logs only go back 30 days to 10 May, so H could have seen 28 May activity if it occurred on the VPN | VPN logs for 28 May–3 June |
Clarifying requests: to H — "Please review the VPN authentication logs for 28 May–3 June for sessions by any account in the attached list (the cloud admin and accounts it could reset) and state whether the 01:12 session used valid credentials or an exploit signature." To C — "Please identify whether the access key created on 28 May was used from any address in the VPN pool or Calder's on-premises ranges."
Possible synthesis after responses (fictional): the 01:12 VPN session authenticated with valid credentials of an account whose password was reset by the phished cloud admin on 30 May. Both reports were partly right: initial access was the phishing (28 May); the VPN session was a later step, not an exploit.
Confidence statement: "We assess with moderate confidence that initial access occurred on 28 May through phishing of a cloud administrator. Supporting evidence: IdP sign-ins from an anonymizing service, an access key created minutes later, and a password reset of the account later used on the VPN. The alternative — an exploited VPN vulnerability on 3 June — is less likely because the VPN session authenticated with valid credentials and no exploit signature was found. Our assessment would change if an earlier compromise of the cloud admin's device were found or if the VPN vendor confirms an authentication-bypass vulnerability affecting our version."
3.9 Common errors and limits
- Accepting the more authoritative-sounding report without checking what it examined.
- Forcing a single narrative before discriminating evidence is obtained.
- Vague requests ("please look again") to experts.
- Confidence words without defined meaning.
Calder's ransomware case shows how the first error happens. The external firm's report H sounded authoritative and arrived first, but its artefacts were VPN appliance logs and a jump-host image; it never saw the IdP. Reconciling it with report C took two precise requests, one to each team, naming dates, accounts and address ranges. A request to "look again" would have produced two longer versions of the same reports.
Exercises — chapter 3
Attempt each exercise before opening the worked results. Every case, organization, person, address and number in an exercise is synthetic unless a cited public source is named.
EX011 — Deconstruct a report. Inputs: Report H summary. Steps: separate artefacts, methods, observations, interpretations and limitations; list what is missing. Rubric: 3 = all five layers, missing items named (tool versions, IdP data not examined, retention); 2 = four layers; 1 = summary; 0 = none.
EX012 — Expert questions. Steps: write five questions to an external firm's lead before accepting their conclusion. Rubric: 3 = scope examined/not examined, methods and versions, alternatives considered, confidence basis, what would change the view; 2 = three; 1 = generic; 0 = none.
EX013 — Precise request. Steps: rewrite "Can the cloud team check the logs again?" into a precise request. Rubric: 3 = specific log source, time window, entities, question, output format and deadline; 2 = missing window or entities; 1 = still vague; 0 = none.
EX014 — Confidence scale. Steps: define a four-level confidence scale for your organization's incident reports with criteria for each level. Rubric: 3 = levels with evidential criteria (independent sources, quality, alternatives excluded), distinct from likelihood terms; 2 = levels without criteria; 1 = words only; 0 = none.
EX015 — Executive version. Steps: rewrite the confidence statement from this chapter's disputed-initial-access worked example for the executive committee in 80 words. Rubric: 3 = conclusion, confidence, implication for decisions, what is being done, what could change it; plain language; 2 = too technical; 1 = drops uncertainty; 0 = none.
Chapter synthesis
Treat specialist reports as layered claims; ask what each expert examined and did not; reconcile disagreements by locating their different evidence; request discriminating evidence precisely; and state conclusions with defined confidence and change conditions.