Chapter 28 — Implementation of evidence capture services
Lesson map
Lesson 28 published. Map, example, EX055.
flowchart TD
N0["Learning outcomes and connection"]
N1["28.1 Requirements for capture"]
N2["28.2 Transfer protocol"]
N3["28.3 Implementation sketch stdlib"]
N4["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4
N4 --> EX["Worked example"]
EX --> Q["Exercise EX055"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["28.1 Requirements for capture"]
N2["28.2 Transfer protocol"]
N3["28.3 Implementation sketch stdlib"]
N4["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX055"]
[Learning outcomes and connection]
|
v
[28.1 Requirements for capture]
|
v
[28.2 Transfer protocol]
|
v
[28.3 Implementation sketch stdlib]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX055]
Learning outcomes and connection
After this chapter you can (1) implement a bounded evidence capture path that reads sources without modifying them; (2) produce per-item checksums and an anchored manifest; (3) log every access and transformation; and (4) recover correctly from interrupted transfers without loss or silent duplication. It builds on Chapters 4, 13, 19 and 22 and prepares Chapters 30, 40 and 47.
28.1 Requirements for capture
A capture service must: open sources read-only (or from snapshots) so collection does not alter them; compute a hash at the source before transfer and again at the destination; write items to storage that the collector can append to but not modify or delete; append each item to a manifest chained by hash and anchored independently (Chapter 19); record an access log entry — who, what, when, why (case ID); and resume interrupted transfers without creating duplicate or partial items that look complete. NIST SP 800-86 emphasises integrity and documentation in collection; RFC 3227 emphasises order of volatility and chain of custody.
28.2 Transfer protocol
A robust design uses content-addressed items: the destination key is the SHA-256 of the content. Transfer proceeds in chunks; each chunk is acknowledged; the destination assembles into a temporary object and only commits (renames into the content-addressed key and appends to the manifest) after the full hash matches the source hash. Retries resend missing chunks. Because the key is the hash, a resent complete item is an idempotent no-op; because commit happens only after verification, a partial item never appears as evidence.
28.3 Implementation sketch (stdlib)
import hashlib, os, json, time
def capture(src_path, dest_dir, manifest, case_id, actor, chunk=1 << 20):
h = hashlib.sha256()
with open(src_path, "rb") as f: # read-only
for block in iter(lambda: f.read(chunk), b""): h.update(block)
digest = h.hexdigest()
final = os.path.join(dest_dir, digest)
if os.path.exists(final): # idempotent resume
return digest, "already-committed"
tmp = final + ".part"
with open(src_path, "rb") as f, open(tmp, "wb") as out:
for block in iter(lambda: f.read(chunk), b""): out.write(block)
check = hashlib.sha256(open(tmp, "rb").read()).hexdigest()
if check != digest:
os.remove(tmp); raise IOError("hash mismatch; source changed or transfer corrupt")
os.replace(tmp, final) # atomic commit
manifest.append({"sha256": digest, "src": src_path, "case": case_id,
"by": actor, "at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())})
return digest, "committed"
The sketch omits network transport and the append-only storage policy, which in production come from the storage service (for example write-once retention). A mismatch between first-read and committed hash usually means the source changed during collection — a finding in itself, recorded rather than retried blindly. Note that the sketch rereads and recopies the whole source on each attempt; chunk-level resume would need per-chunk acknowledgement state at the destination. The .part suffix is what keeps an incomplete copy out of the manifest until os.replace commits it.
28.4 Worked example (synthetic): interrupted transfer at HVSS
An HVSS collector copies a 2 GB log archive from a branch server; the link drops at 60%. On resume, the destination has ….part but no committed object and no manifest entry: the evidence set correctly shows the item as not yet collected. After resume, the hash matches; commit and manifest append occur once. A second, accidental re-run returns already-committed. An analyst querying the case sees one item, its hash, the collector identity, the access log and two transfer attempts — a complete and honest history.
28.5 Errors and limits
- Hashing only at the destination.
- Partial files visible as evidence.
- Overwriting by filename instead of content address.
- Collector credentials that can delete stored evidence.
The interrupted HVSS transfer in the worked example shows why each of these guards matters. Because the 2 GB archive was hashed at the branch server before transfer, a corrupt copy could be told apart from a source that changed. The 60% .part file never appeared as evidence because commit waited for a matching hash. Content addressing turned the accidental re-run into already-committed instead of a second copy. A collector able to delete stored items would undermine the whole access history.
Exercises — chapter 28
EX055 — Run and break. Objective: verify behaviour. Inputs: the sketch. Steps: create a synthetic file, capture it twice, then simulate source change between hash and copy (append a byte after hashing via a test hook); record results. Expected evidence: code and outputs. Rubric: 3 = shows committed, already-committed and mismatch error with no manifest entry for the failed case; 2 = two; 1 = one; 0 = none.
EX056 — Access log. Objective: design accountability. Inputs: capture service. Steps: specify the access-log record fields and where it is stored so that the collector cannot alter it. Expected evidence: field list and storage design. Rubric: 3 = actor, case, source, purpose, time, result, hash; separate append-only store with independent admin; 2 = partial; 1 = vague; 0 = none.
Chapter synthesis
Evidence capture is a small protocol: read-only sources, hash before transfer, verify before atomic commit, content-addressed idempotent storage, anchored manifest and independent access logs. Interrupted transfers then leave honest gaps, never false evidence.
Continue studying — chapter 28 of 50
← Chapter 27 · Course chapter index · Chapter 29 →
Continue to the next authored lesson.