Chapter 17 — Cryptography concepts
Lesson map
Lesson 17 published. Map, example, EX049.
flowchart TD
N0["Learning outcomes"]
N1["17.1 Core primitives"]
N2["17.2 Protecting AI artefacts"]
N3["17.3 What cryptography cannot do"]
N4["17.4 Keys"]
N5["17.6b Verifying signatures in practice"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX049"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["17.1 Core primitives"]
N2["17.2 Protecting AI artefacts"]
N3["17.3 What cryptography cannot do"]
N4["17.4 Keys"]
N5["17.6b Verifying signatures in practice"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX049"]
[Learning outcomes]
|
v
[17.1 Core primitives]
|
v
[17.2 Protecting AI artefacts]
|
v
[17.3 What cryptography cannot do]
|
v
[17.4 Keys]
|
v
[17.6b Verifying signatures in practice]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX049]
Learning outcomes
You will (1) explain hashing, symmetric and public-key encryption and digital signatures; (2) apply them to protect AI artefacts and data; (3) explain what cryptography cannot do for AI security; and (4) verify model integrity with signed manifests.
17.1 Core primitives
Hash functions (such as SHA-256) map data to a short digest; changing one bit changes the digest. Symmetric encryption (such as AES) protects data with a shared key. Public-key cryptography lets anyone encrypt to, or verify signatures from, a key holder. Digital signatures prove who created data and that it is unchanged.
In LCH's systems, hashes check model files, symmetric encryption protects stored conversation logs, public-key cryptography secures TLS to the model provider, and signatures prove that DocSort releases come from the vendor. Each primitive answers a different question, and using the wrong one leaves a gap: a hash alone shows a file is unchanged but not who produced it.
17.2 Protecting AI artefacts
Encrypt training data and model weights at rest; use TLS in transit. Publish SHA-256 digests of model files in a manifest and sign the manifest, so recipients can verify both integrity and origin before loading. This addresses supply-chain risks (OWASP LLM03) such as tampered or backdoored models (Gu et al., arXiv:1708.06733).
"""Lab H: integrity of model artefacts with SHA-256 manifests (stdlib only).
Creates synthetic 'artefact' bytes, records a manifest, then detects a
one-byte tampering and a swapped file. No real models are involved."""
import hashlib, json
files = {"model.bin": bytes(range(256)) * 40, "tokenizer.json": b'{"vocab": ["a", "b"]}',
"config.yaml": b"layers: 4\nheads: 8\n"}
manifest = {n: hashlib.sha256(b).hexdigest() for n, b in files.items()}
print(json.dumps({k: v[:16] for k, v in manifest.items()}, indent=1))
received = dict(files)
tampered = bytearray(received["model.bin"]); tampered[5000] ^= 1
received["model.bin"] = bytes(tampered)
received["config.yaml"] = b"layers: 4\nheads: 8\ntrust_remote_code: true\n"
for n, b in sorted(received.items()):
ok = hashlib.sha256(b).hexdigest() == manifest[n]
print(f"{n:15s} {'OK' if ok else 'MISMATCH'}")
Expected output:
{
"model.bin": "e96760a87768717b",
"tokenizer.json": "f63d2ddabc61f2d9",
"config.yaml": "e5bbb570ca0fcb87"
}
config.yaml MISMATCH
model.bin MISMATCH
tokenizer.json OK
Lab H checks integrity but not origin, because an attacker who can change the files might also change an unsigned manifest. Signing closes that gap, since the key is not on the server.
17.3 What cryptography cannot do
Encryption does not stop an authorised query from extracting information from a model, does not prevent prompt injection, and does not make a poisoned dataset clean — it only proves the poisoned data is the data you signed. Cryptography protects data and artefacts; behaviour needs other controls.
For Ask Lumen, every message travels encrypted and every model file is verified, yet a patient can still type an injection and a partner document can still carry hostile text. Those risks need the policy point, labelling and monitoring. Rina's architecture note lists which threats each layer addresses, so nobody assumes encryption covers more than it does.
17.4 Keys
Security depends on keys: store them in key-management services, restrict access, rotate them and never place them in prompts or code repositories.
The API key for LCH's model provider is a clear example: anyone holding it can run queries billed to LCH and possibly read logs. LCH stores it in the cloud key-management service, gives only the application server access, rotates it every quarter and scans repositories for accidental commits. A leaked key is revoked first, then investigated, and the logs reviewed.
17.5 Worked example (synthetic LCH case)
LCH's DocSort vendor began signing releases. Rina configured the deployment script to fetch the vendor's public key from LCH's own trusted store (not from the release server), verify the signature on the manifest, then verify each file's digest. A test release with a modified file was rejected.
Fetching the key from LCH's own store matters because an attacker who controls the release server could otherwise supply both a malicious file and a matching key. The rejected test release confirms the check works end to end.
17.6 Errors and limits
- Fetching digests and keys from the same place as the files.
- Assuming encryption prevents model leakage.
- Keys in source code.
Fetching digests and keys from the same place as the files means one compromise defeats the check, which is why LCH keeps the DocSort key in its own store. Encryption does not prevent a model from revealing training data through its answers. Keys in source code end up in repositories and backups where many people can read them, so they are scanned for and rotated when found.
17.6b Verifying signatures in practice
Signing DocSort's model file helps only if the loader checks the signature every time and refuses to start on failure. LCH tested this by loading a copy with one byte changed: the service refused, logged the event and kept running the previous verified version.
Keeping the previous verified version running avoided an outage while the problem was investigated, which made refusing safe. The vendor then reissued a correctly signed release. Staff noticed no change in DocSort while the old version kept sorting documents.
Exercises
- EX049 — Primitives. Match: integrity of a file; origin of a manifest; confidentiality of stored data. Rubric: 3 = hash, signature, encryption; 2 = two; 1 = one; 0 = none.
- EX050 — Key source. Why fetch the vendor's public key from LCH's own store? Rubric: 3 = attacker controlling release server could replace both files and key; 2 = partial; 1 = vague; 0 = wrong.
- EX051 — Limits. Name two AI risks cryptography does not address. Rubric: 3 = prompt injection, extraction via queries, poisoning of signed data, evasion; 2 = one; 1 = vague; 0 = wrong.
Synthesis
Cryptography makes AI artefacts and data verifiable and confidential, but it cannot make a model behave; it is one layer among many.
Continue studying — chapter 17 of 40
← Chapter 16 (draft) · Course chapter index · Chapter 18 (draft) →