Chapter 30 — Verification laboratory I
Lesson map
Lesson 30 published. Map, example, EX088.
flowchart TD
N0["Learning outcomes"]
N1["30.1 Verification and validation"]
N2["30.2 Test vectors"]
N3["Lab — test vectors"]
N4["30.3 Reading the result"]
N5["30.4 Beyond vectors"]
N6["30.5 Recording the run"]
N7["30.6 Failed vectors"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX088"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["30.1 Verification and validation"]
N2["30.2 Test vectors"]
N3["Lab — test vectors"]
N4["30.3 Reading the result"]
N5["30.4 Beyond vectors"]
N6["30.5 Recording the run"]
N7["30.6 Failed vectors"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX088"]
[Learning outcomes]
|
v
[30.1 Verification and validation]
|
v
[30.2 Test vectors]
|
v
[Lab — test vectors]
|
v
[30.3 Reading the result]
|
v
[30.4 Beyond vectors]
|
v
[30.5 Recording the run]
|
v
[30.6 Failed vectors]
|
v
[Worked example] --> [Exercise EX088]
Learning outcomes
You will (1) verify a hashing component against published test vectors; (2) distinguish verification from validation; (3) record a verification run; (4) handle a failed vector.
30.1 Verification and validation
Verification asks whether the component meets its specification. Validation asks whether the whole method is fit for its forensic purpose. ISO/IEC 27041 addresses assuring that methods are suitable and adequate; verification of components is one input.
A hash routine can pass every component test and the method can still be unsuitable, for example if the lab's procedure lets examiners skip verification. Validation tests the whole chain under realistic conditions: real hardware, real procedures and realistic evidence. Priya's report keeps the two separate, listing verified components first and then validation evidence for the method as used.
30.2 Test vectors
Standards for hash functions publish example inputs with expected digests. NIST's example values for SHA-256 include the message "abc". The empty message has a widely published SHA-256 digest too. Matching them shows the implementation computes the algorithm correctly for those inputs.
Test vectors are small, fixed inputs with known outputs, so a passing result shows that the implementation agrees with the standard on those inputs. They do not cover every input, but an algorithm implemented wrongly almost always fails at least one of them. Priya copies the expected values from the source document into her test file and records where they came from.
Lab — test vectors
"""Validate the hashing library against published test vectors."""
import hashlib
vec = {b"abc": "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad",
b"": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}
ok = 0
for m, d in vec.items():
r = hashlib.sha256(m).hexdigest() == d; ok += r
print(repr(m), "PASS" if r else "FAIL")
print(f"{ok}/{len(vec)} vectors passed")
Run with python3 lab_vectors.py. Verified output:
b'abc' PASS
b'' PASS
2/2 vectors passed
30.3 Reading the result
Both vectors pass: "abc" and the empty message produce the expected digests, 2/2. A single failure would stop the release until explained.
Two passing vectors are a small sample, so Priya treats them as a basic correctness check rather than a full assurance. A failure would be serious enough to block the release on its own, because any mismatch on a published vector means either the code or the test is wrong, and the lab cannot use EvidenceDesk until it knows which.
30.4 Beyond vectors
Vectors test the algorithm, not the surrounding code that reads files. Priya adds tests that hash synthetic files of known content via EvidenceDesk's own read path and compare with the library result.
A file-reading bug might skip the last partial chunk, read past the end or open files in text mode, which changes line endings on some systems. Tests with files of 0 bytes, 1 byte, exactly one chunk and one chunk plus one byte check these cases. Each EvidenceDesk result is compared with a direct library call on the same bytes.
30.5 Recording the run
The lab notebook records date, tester, code version, Python version, vectors used and results. The record is signed and stored with the release.
The signature is the tester's own, with the date. The record is stored with the release, so anyone examining a past case can find which tests the version in use had passed. If a test is later found to be flawed, the records show which releases relied on it, and those releases can be reassessed. Unsigned records do not count.
30.6 Failed vectors
If a vector fails, suspect the test first (typo in the expected value), then the code. Both checks are recorded.
Typos in expected values are common: one wrong hex digit among sixty-four is easy to miss. Priya checks the expected value against a second independent source before suspecting her code. If the test is right and the code is wrong, the defect is logged, the release is blocked and the fix includes a new regression test for the case that failed.
30.7 Repeat on release
Vectors run automatically on every build, so a regression is caught immediately.
The build script runs the vectors before packaging and refuses to produce a release artefact if any fail. The results are attached to the build record. Running them on every build, not just before releases, means a regression introduced on Tuesday is noticed on Tuesday rather than weeks later when the change is harder to trace. Developers can also run the vectors locally in seconds before pushing a change, so most failures never reach the shared build.
Independent check
Priya also compares EvidenceDesk digests with those from a second, independently written tool on twenty synthetic files. Agreement across tools adds confidence beyond the vectors.
Worked example
A classmate's run fails the "abc" vector. Investigation shows a trailing newline was hashed — the input was read from a text file. Fixing the test input restores the pass; the record notes the cause.
Synthetic case. The classmate's run is invented.
Errors and limits
Passing vectors does not show large-file handling or file-reading code is correct; separate tests do.
Exercises
- EX088 — V and V. Distinguish verification from validation. Rubric: 3 = meets spec vs fit for forensic purpose; 2 = partial; 1 = same; 0 = none.
- EX089 — Newline failure. What caused the failed vector? Rubric: 3 = trailing newline from text input; 2 = partial; 1 = library bug; 0 = none.
- EX090 — Vector limits. What do vectors not test? Rubric: 3 = file reading path, large files; 2 = partial; 1 = everything tested; 0 = none.
Synthesis
Vectors verify the algorithm; file-path tests and recorded runs extend trust to the component.
Continue studying — chapter 30 of 50
← Chapter 29 (draft) · Course chapter index · Chapter 31 (draft) →