Chapter 9 — Evidence and measurement
Lesson map
Lesson 09 published. Map, example, EX025.
flowchart TD
N0["Learning outcomes"]
N1["9.1 Logs as evidence"]
N2["Lab — Handshake failure summary"]
N3["9.2 Reading the summary"]
N4["9.3 Reporting clearly"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX025"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["9.1 Logs as evidence"]
N2["Lab — Handshake failure summary"]
N3["9.2 Reading the summary"]
N4["9.3 Reporting clearly"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX025"]
[Learning outcomes]
|
v
[9.1 Logs as evidence]
|
v
[Lab — Handshake failure summary]
|
v
[9.2 Reading the summary]
|
v
[9.3 Reporting clearly]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX025]
Learning outcomes
You will (1) read TLS failure logs to find causes; (2) count and classify events reliably; (3) report evidence clearly to engineers and managers.
9.1 Logs as evidence
When citizens report "the site is broken", logs tell what actually happened. The council's web gateway records failed TLS handshakes with a client address, a reason code and the target host. Rosa learns to read these logs before guessing. Reason codes such as certificate_expired, protocol_version, unknown_ca and handshake_timeout point to different fixes. She writes a small parser using only the standard library to summarise a morning's failures, working on a synthetic sample with addresses from a private range.
Lab — Handshake failure summary
"""Count reasons in a synthetic TLS handshake-failure log from a city web gateway."""
from collections import Counter
LOG = """\
2026-03-02T08:01Z fail client=10.2.0.14 reason=certificate_expired host=permits
2026-03-02T08:03Z fail client=10.2.0.31 reason=protocol_version host=legacy-gis
2026-03-02T08:07Z fail client=10.2.0.14 reason=certificate_expired host=permits
2026-03-02T08:11Z fail client=10.2.7.90 reason=unknown_ca host=staff-vpn
2026-03-02T08:12Z fail client=10.2.0.55 reason=certificate_expired host=permits
2026-03-02T08:20Z fail client=10.2.0.31 reason=protocol_version host=legacy-gis
2026-03-02T08:24Z fail client=10.2.3.12 reason=handshake_timeout host=permits"""
def main():
rows = [dict(kv.split("=") for kv in ln.split()[2:]) for ln in LOG.splitlines()]
for (r, h), n in Counter((x["reason"], x["host"]) for x in rows).most_common():
print(f"{n} {r:<20} {h}")
print("distinct clients hitting expired cert:", len({x["client"] for x in rows if x["reason"] == "certificate_expired"}))
if __name__ == "__main__":
main()
Run with python3 lab_tlslog.py. Verified output:
3 certificate_expired permits
2 protocol_version legacy-gis
1 unknown_ca staff-vpn
1 handshake_timeout permits
distinct clients hitting expired cert: 2
9.2 Reading the summary
Of seven failures, three are certificate_expired on the permits host from two distinct clients, two are protocol_version on legacy-gis from one old client, one is unknown_ca on staff-vpn and one is a timeout on permits. Rosa concludes that the permits certificate is the urgent problem, affecting citizens; the legacy GIS failures come from one outdated client that cannot negotiate a modern TLS version; and the unknown CA error suggests a staff device missing the internal root. She separates each into its own ticket with the evidence attached, rather than one vague "TLS problem" ticket.
9.3 Reporting clearly
For engineers Rosa reports counts, hosts, reason codes and time ranges. For managers she writes two sentences: citizens could not renew permits for about twenty minutes because a certificate expired, and it was renewed at 08:30; two minor issues affect a GIS client and one staff device. She avoids exaggerating and includes when evidence is incomplete, such as failures not logged when the gateway was overloaded. Clear, audience-appropriate reporting helps the council act quickly and builds trust in IT's information. She closes each report with one recommended next step, so readers who skim still know exactly what to do.
Worked example
Rosa parses a synthetic failure log, identifies the expired permits certificate as urgent, splits three issues into tickets and writes separate summaries for engineers and managers.
Synthetic case. The log is invented.
Errors and limits
Logs may miss events during overload. Reason codes depend on server software and can be ambiguous.
Exercises
- EX025 — Reason codes. Name three TLS failure reason codes and what they suggest. Rubric: 3 = e.g. certificate_expired (renew), protocol_version (old client), unknown_ca (missing root);; 2 = two;; 1 = one;; 0 = none.
- EX026 — Urgent issue. Which issue did Rosa rank most urgent and why? Rubric: 3 = expired permits certificate, affecting citizens (3 failures, 2 clients);; 2 = issue;; 1 = other;; 0 = none.
- EX027 — Manager summary. What did the manager summary say? Rubric: 3 = citizens blocked about twenty minutes by expired certificate, renewed 08:30; two minor issues;; 2 = partial;; 1 = technical detail;; 0 = none.
Synthesis
Reading logs carefully, classifying events and reporting to each audience turns cryptographic failures into actionable evidence.
Continue studying — chapter 09 of 30
← Chapter 08 (draft) · Course chapter index · Chapter 10 (draft) →