Chapter 4 — Scripting and data formats
Lesson map
Lesson 04 published. Map, example, EX013.
flowchart TD
N0["Learning outcomes"]
N1["4.1 Encoding, hashing, encryption"]
N2["4.2 Integrity manifests"]
N3["Lab — SHA-256 manifest check"]
N4["4.3 Reading PEM files"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX013"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["4.1 Encoding, hashing, encryption"]
N2["4.2 Integrity manifests"]
N3["Lab — SHA-256 manifest check"]
N4["4.3 Reading PEM files"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX013"]
[Learning outcomes]
|
v
[4.1 Encoding, hashing, encryption]
|
v
[4.2 Integrity manifests]
|
v
[Lab — SHA-256 manifest check]
|
v
[4.3 Reading PEM files]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX013]
Learning outcomes
You will (1) distinguish encoding, hashing and encryption in data you meet daily; (2) build a SHA-256 integrity manifest with Python; (3) read PEM and Base64 safely.
4.1 Encoding, hashing, encryption
Tomás often sees long strings in configuration files and must tell them apart. Base64 and hex are encodings: they change representation and anyone can reverse them without a key. A hash such as SHA-256, specified in FIPS 180-4, maps any input to a fixed 32-byte digest and cannot be reversed, but anyone can compute it, so it proves integrity only when the expected digest is itself trusted. Encryption needs a key to reverse. A colleague once called a Base64 password encrypted; Tomás now explains that decoding it takes one command and no secret.
4.2 Integrity manifests
The nightly partner files arrive by SFTP, and occasionally a truncated transfer goes unnoticed. Priya suggests a manifest: the sender lists each file name with its SHA-256 digest, and the receiver recomputes and compares. Tomás writes a small script that does this on synthetic data. A manifest catches accidental corruption reliably. Against an attacker who can change both the file and the manifest it proves nothing, so in production the manifest itself must be signed or delivered over an authenticated channel. The lab below shows both the success case and a detected change.
Lab — SHA-256 manifest check
# Synthetic SHA-256 manifest check (FIPS 180-4 via hashlib). Teaching only.
import hashlib
files = {"routes_mon.csv": b"depot,route\nN1,12\nN2,7\n",
"routes_tue.csv": b"depot,route\nN1,13\nN2,7\n"}
manifest = {n: hashlib.sha256(d).hexdigest() for n, d in files.items()}
files["routes_tue.csv"] = b"depot,route\nN1,18\nN2,7\n" # simulated change in transit
for name, data in files.items():
ok = hashlib.sha256(data).hexdigest() == manifest[name]
print(f"{name}: {'OK' if ok else 'CHANGED'} {manifest[name][:16]}...")
Run with python3 lab_manifest.py. Verified output:
routes_mon.csv: OK 71e12cbcb1238386...
routes_tue.csv: CHANGED 03db2f9cd00a57f9...
4.3 Reading PEM files
Certificates and keys usually arrive as PEM: Base64 text between BEGIN and END lines. Tomás learns to look at the header before anything else. BEGIN CERTIFICATE is public and safe to share; BEGIN PRIVATE KEY or BEGIN RSA PRIVATE KEY is secret and must never be pasted into a ticket, a chat or an online decoder. When he needs to inspect a certificate he uses local tools rather than websites. Decoding PEM shows DER bytes, which tools like openssl x509 -text turn into readable fields such as subject, issuer and validity dates.
Worked example
Tomás runs the manifest lab, sees one partner file flagged as changed, and explains to Priya why a signed manifest would be needed against a deliberate attacker.
Synthetic case. File names and contents in the lab are synthetic.
Errors and limits
Hash manifests do not authenticate the sender. SHA-1 and MD5 manifests are obsolete for security purposes; use SHA-256 or stronger.
Exercises
- EX013 — Base64 password. A config holds a Base64 string labelled encrypted password. What is wrong? Rubric: 3 = Base64 is reversible encoding with no key; it is not encryption;; 2 = says not encrypted;; 1 = agrees it is encrypted;; 0 = none.
- EX014 — Manifest result. In the lab, which file was flagged and why? Rubric: 3 = routes_tue.csv, its digest no longer matched after one character changed;; 2 = names file;; 1 = no file;; 0 = none.
- EX015 — Manifest limit. What attack does an unsigned manifest fail to stop? Rubric: 3 = an attacker who replaces both the file and the manifest;; 2 = mentions attacker;; 1 = says none;; 0 = none.
- EX016 — PEM header check. Which PEM header must never be pasted into a ticket? Rubric: 3 = any PRIVATE KEY header, because it is secret;; 2 = mentions private;; 1 = certificate header;; 0 = none.
Synthesis
Knowing whether data is encoded, hashed or encrypted tells a technician what it can prove and who can reverse it.
Continue studying — chapter 04 of 18
← Chapter 03 (draft) · Course chapter index · Chapter 05 (draft) →