Chapter 7 — Probability and statistics
Lesson map
Lesson 07 published. Map, example, EX019.
flowchart TD
N0["Learning outcomes"]
N1["7.1 Evaluative reporting"]
N2["7.2 A digital example"]
N3["Lab — likelihood ratio"]
N4["7.3 Reading the result"]
N5["7.4 The prosecutor's fallacy"]
N6["7.5 Where probabilities come from"]
N7["7.6 Uncertainty in estimates"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX019"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["7.1 Evaluative reporting"]
N2["7.2 A digital example"]
N3["Lab — likelihood ratio"]
N4["7.3 Reading the result"]
N5["7.4 The prosecutor's fallacy"]
N6["7.5 Where probabilities come from"]
N7["7.6 Uncertainty in estimates"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX019"]
[Learning outcomes]
|
v
[7.1 Evaluative reporting]
|
v
[7.2 A digital example]
|
v
[Lab — likelihood ratio]
|
v
[7.3 Reading the result]
|
v
[7.4 The prosecutor's fallacy]
|
v
[7.5 Where probabilities come from]
|
v
[7.6 Uncertainty in estimates]
|
v
[Worked example] --> [Exercise EX019]
Learning outcomes
You will (1) apply conditional probability to evidence; (2) compute a likelihood ratio; (3) see how priors affect conclusions; (4) avoid the prosecutor's fallacy.
7.1 Evaluative reporting
Forensic science increasingly frames findings as how much more probable the evidence is under one proposition than another. This is the likelihood ratio: LR = P(E | H1) / P(E | H2).
The two propositions must be stated before the evidence is assessed, and each must be a realistic account that one side might put forward. The likelihood ratio then measures how strongly the evidence favours one account over the other. An LR above one supports H1 and below one supports H2; an LR near one means the evidence hardly helps to distinguish them, which is a useful thing for a court to hear.
7.2 A digital example
E: a "recent documents" entry exists for report.docx. H1: the user opened the file. H2: the file was only synced by a cloud client and never opened. Synthetic testing estimates P(E | H1) = 0.90 and P(E | H2) = 0.05.
To estimate those probabilities, Priya sets up synthetic machines with the same operating system and cloud client. In one group a test user opens the document. In the other the file only arrives by synchronisation. She then checks for the entry. Twenty runs per group produce the 0.90 and 0.05 figures. The test design, including the version numbers, is recorded as part of the method.
Lab — likelihood ratio
"""Likelihood ratio for a synthetic file-presence finding."""
p_e_h1 = 0.90 # P(artefact | user opened file)
p_e_h2 = 0.05 # P(artefact | file only synced, never opened)
lr = p_e_h1 / p_e_h2
for prior in (0.1, 0.5):
po = prior / (1 - prior) * lr
print(f"prior={prior:.1f} LR={lr:.0f} posterior={po/(1+po):.3f}")
Run with python3 lab_lr.py. Verified output:
prior=0.1 LR=18 posterior=0.667
prior=0.5 LR=18 posterior=0.947
7.3 Reading the result
LR = 18: the entry is 18 times more probable if the user opened the file. With a prior of 0.1 the posterior is 0.667; with 0.5 it is 0.947. The evidence is the same; the conclusion depends on the prior, which is for the court, not the examiner, to supply.
The arithmetic uses odds. A prior of 0.1 is odds of 1 to 9, and multiplying by 18 gives odds of 2 to 1, which is a probability of 0.667. A prior of 0.5 is odds of 1 to 1, which become 18 to 1, or about 0.947. Showing both cases makes the point that the examiner's contribution is the factor of 18, while the starting odds belong to the fact-finder.
7.4 The prosecutor's fallacy
Saying "there is only a 5% chance the user did not open it" confuses P(E | H2) with P(H2 | E). Examiners report the LR and leave priors to fact-finders.
The mirror error, sometimes called the defence fallacy, plays down evidence by saying that many people could have produced it, ignoring the other evidence that narrows the field. Both fallacies swap one conditional probability for another. Priya practises catching them by rewriting each draft sentence in the form 'probability of this evidence if…' and checking that every claim fits that pattern.
7.5 Where probabilities come from
The 0.90 and 0.05 must come from tests on systems like the one examined; a different OS version may behave differently.
A recent-documents entry might be written differently after an update, or only when a particular setting is on. If the examined machine runs a version that was not tested, the LR is extrapolated and the report has to say so. Priya keeps a table listing each tested configuration and the probability estimated for it, so a reader can see exactly how close the test conditions were to the case.
7.6 Uncertainty in estimates
With 20 test runs, 0.90 could plausibly be 0.70–0.97; report how many runs supported each figure.
With 18 positives out of 20 the point estimate is 0.90, but a Wilson interval runs from about 0.70 to 0.97. With 1 out of 20 for H2 the estimate is 0.05 and the interval is very wide in relative terms, perhaps 0.01 to 0.24. Since the LR divides one figure by the other, its range is wide too, roughly 3 to 97. Reporting that range is more honest than a single 18. More runs for H2 would narrow the range fastest, because the small denominator dominates the uncertainty.
Sources of test data
The probabilities come from controlled synthetic runs: twenty sessions in which a test user opened the file, and twenty in which a cloud client only synced it. Each run uses a fresh virtual machine with the same OS build, so the comparison is fair and others can repeat it.
Worked example
Priya drafts: "The entry is about 18 times more likely if the file was opened than if it was only synced, based on 20 synthetic tests on the same OS version."
Synthetic case. The probabilities are invented for teaching.
Errors and limits
An LR addresses only the two propositions compared; a third explanation needs its own probability.
Exercises
- EX019 — LR value. Compute the LR from 0.90 and 0.05. Rubric: 3 = 18 with formula; 2 = number; 1 = 0.85; 0 = none.
- EX020 — Fallacy. What is wrong with "only 5% chance the user did not open it"? Rubric: 3 = transposes conditional; 2 = partial; 1 = vague; 0 = none.
- EX021 — Prior effect. Why do posteriors 0.667 and 0.947 differ for the same evidence? Rubric: 3 = different priors; examiner reports LR; 2 = partial; 1 = error; 0 = none.
Synthesis
Likelihood ratios express evidential strength honestly; priors and conclusions belong to the court.
Continue studying — chapter 07 of 50
← Chapter 06 (draft) · Course chapter index · Chapter 08 (draft) →