Chapter 25 — Laboratory I
Lesson map
Lesson 25 published. Map, example, EX049.
flowchart TD
N0["Learning outcomes"]
N1["25.1 Why test implementations"]
N2["25.2 Practical activity — Lab V"]
N3["25.3 Lab records"]
N4["25.4 Interpreting results"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX049"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["25.1 Why test implementations"]
N2["25.2 Practical activity — Lab V"]
N3["25.3 Lab records"]
N4["25.4 Interpreting results"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX049"]
[Learning outcomes]
|
v
[25.1 Why test implementations]
|
v
[25.2 Practical activity — Lab V]
|
v
[25.3 Lab records]
|
v
[25.4 Interpreting results]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX049]
Learning outcomes
You will (1) run known-answer tests (KATs) and negative tests; (2) explain why validation programs test implementations; (3) build a reproducible lab record; and (4) interpret pass and fail results correctly.
25.1 Why test implementations
Standards define algorithms; implementations can still be wrong. Known-answer tests run an implementation on fixed inputs and compare with published outputs; negative tests feed malformed inputs and expect rejection. NIST's Cryptographic Algorithm Validation Program applies such tests to implementations of FIPS 203, 204 and 205. Students cannot validate a production library, but they can learn the habit with primitives from the standard library.
The point of the exercise is to separate two claims that are easy to blur: that an algorithm is sound, and that a given piece of code computes it correctly. HCU's developers rely on the first from the standards process and must establish the second themselves for every library they deploy.
25.2 Practical activity — Lab V
# Lab V (ch 25): known-answer and negative tests - the habit behind validating PQC libraries.
import hashlib
KATS = [ # (function, input, expected hex prefix) - published test values for SHA-3 / SHAKE on empty input
("sha3_256", b"", "a7ffc6f8bf1ed76651c14756a061d662f580ff4de43b49fa82d80a4b80f8434a"),
("shake_128/32", b"", "7f9c2ba4e88f827d616045507605853ed73b8093f6efbc88eb1a6eacfa66ef26"),
]
def run(name, data):
if name == "sha3_256": return hashlib.sha3_256(data).hexdigest()
return hashlib.shake_128(data).hexdigest(32)
for name, data, want in KATS:
print(f"KAT {name:13s}", "PASS" if run(name, data) == want else "FAIL")
SIZES = {"ML-KEM-512": 800, "ML-KEM-768": 1184, "ML-KEM-1024": 1568} # encapsulation-key bytes, FIPS 203
def validate_ek(param, ek):
if len(ek) != SIZES[param]: raise ValueError(f"{param}: expected {SIZES[param]} bytes, got {len(ek)}")
return True
for param, n in [("ML-KEM-768", 1184), ("ML-KEM-768", 1183), ("ML-KEM-512", 1184)]:
try: validate_ek(param, bytes(n)); print(f"negative/positive test {param} len {n}: accepted")
except ValueError as e: print(f"negative/positive test {param} len {n}: rejected ({e})")
Expected output:
KAT sha3_256 PASS
KAT shake_128/32 PASS
negative/positive test ML-KEM-768 len 1184: accepted
negative/positive test ML-KEM-768 len 1183: rejected (ML-KEM-768: expected 1184 bytes, got 1183)
negative/positive test ML-KEM-512 len 1184: rejected (ML-KEM-512: expected 800 bytes, got 1184)
The hash KATs pass because Python's SHA-3 matches published values; the length checks show one acceptance and two rejections. A real ML-KEM decoder also checks coefficient ranges as FIPS 203 requires.
Both kinds of test are needed. The KATs confirm that correct inputs give the published outputs; the negative tests confirm that inputs of the wrong size are refused rather than processed. Note the third case: a key of the right length for ML-KEM-768 is still rejected when the caller claims ML-KEM-512, because the parameter set must match. Many real interoperability bugs come from exactly this kind of mismatch between a label and the bytes that accompany it.
25.3 Lab records
A lab record states the question, environment (OS, Python and library versions), inputs, exact commands, outputs and interpretation. Store it with the code so that anyone can rerun it. If a library update changes a result, the record shows what changed.
Good records also note what was not tested, so that a later reader does not assume coverage that never existed. For HCU, the lab record for each cryptographic library sits beside its configuration in version control, and the change ticket for any library upgrade links to the rerun record. An auditor can then trace a production version back to the evidence that it passed.
25.4 Interpreting results
A passing KAT shows that the implementation matches the reference on those inputs; it does not show constant-time behaviour, correct key handling or safe integration. A failing test may reveal a bug in the implementation, the test vector or the harness. Include a negative control — a deliberately wrong implementation that the test must catch — to confirm the harness can fail.
In Lab V the rejected key lengths act as a simple version of that control. If the control passes, the harness itself is broken and every earlier result is in doubt.
25.5 Worked example
An HCU developer integrates a new PQC library. Their lab record: library version, test vectors from the vendor's published KAT files, a harness that compares outputs, and a negative control that flips one byte of an expected output to check the harness reports failure. All KATs pass; the negative control fails as expected. The record notes that side-channel behaviour was not tested and is covered by the library's own evaluation.
The developer also records where the vectors came from and when they were downloaded, so that a later update to the vendor files can be checked against the version used.
25.6 Common errors and limits
- Treating KAT success as proof of security.
- Harnesses that cannot fail.
- Unrecorded versions, making results irreproducible.
- Running only positive tests, so that malformed inputs are never exercised.
- Downloading test vectors from an unofficial mirror without checking their origin.
- Rerunning nothing after a library upgrade because the earlier run passed.
- Testing a library in isolation and assuming the integration behaves the same, when the calling code may pass the wrong parameter set or mishandle errors.
Each of these leaves a gap that the lab record should make visible rather than hide.
Exercises
- EX049 — Negative control. Describe a negative control for Lab V's SHA3-256 KAT. Rubric: 3 = deliberately wrong input or expected value that must produce FAIL.
- EX050 — Limits. List three properties a passing KAT does not establish. Rubric: 3 = side channels, key management, integration/protocol use (or equivalents).
Synthesis
Known-answer and negative tests, recorded reproducibly and checked with negative controls, are the basic evidence that an implementation matches its standard — and nothing more.
Continue studying — chapter 25 of 40
← Chapter 24 (draft) · Course chapter index · Chapter 26 (draft) →