Lesson map
Lesson 25 published. Map, example, EX073.
flowchart TD
N0["Learning outcomes and connection"]
N1["25.1 Why timelines are hard"]
N2["25.2 Clock offset estimation"]
N3["25.3 Ordering uncertainty"]
N4["25.4 The lab synthetic"]
N5["25.5 Reading the result"]
N6["25.6 Writing the timeline entry"]
N7["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX073"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["25.1 Why timelines are hard"]
N2["25.2 Clock offset estimation"]
N3["25.3 Ordering uncertainty"]
N4["25.4 The lab synthetic"]
N5["25.5 Reading the result"]
N6["25.6 Writing the timeline entry"]
N7["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX073"]
[Learning outcomes and connection]
|
v
[25.1 Why timelines are hard]
|
v
[25.2 Clock offset estimation]
|
v
[25.3 Ordering uncertainty]
|
v
[25.4 The lab synthetic]
|
v
[25.5 Reading the result]
|
v
[25.6 Writing the timeline entry]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX073]
Learning outcomes and connection
You will be able to (1) normalize timestamps from sources with different offsets; (2) join events across identity and application logs; (3) annotate each timeline entry with its source reliability and uncertainty; and (4) reproduce every transformation from raw input to final timeline. This laboratory applies Chapters 8, 15, 17 and 22. The timeline it produces is reused in the recovery and verification chapters that follow.
25.1 Why timelines are hard
A timeline seems simple — sort events by time — but each source has its own clock, time zone, precision and meaning of "time" (event occurred, event logged, event ingested). Sorting raw strings from different sources produces a false order. The laboratory method is: (1) preserve raw records and hash them; (2) document each source's clock and offset; (3) normalize to UTC with a note; (4) join on stable identifiers; (5) mark uncertain orderings explicitly; (6) write the timeline with a pointer from every entry to its raw record.
25.2 Clock offset estimation
When two sources record the same event (for example a sign-in seen by the IdP and the application's session start), their difference estimates the relative offset. Use several shared events and take the median to reduce the effect of processing delays. If shared events are unavailable, state the offset as unknown and widen the uncertainty of ordering.
In Lab 25 the two shared sessions give offsets of 33 and 32 seconds, so the median is about thirty-two and a half seconds and the spread is one second. Two pairs are enough to illustrate the method but thin as evidence; adding a third shared event, if one exists in the bundle, would test whether the clock is steady or drifting.
25.3 Ordering uncertainty
If two events from different sources are closer in time than the offset uncertainty, their order is not established. Mark them as "order uncertain" rather than choosing one.
In Lab 25 the uncertainty is half the spread plus five seconds, roughly five and a half seconds. The fin-04 sign-in and its application session start fall within that window, as do the two rcp-02 events, so the program flags both pairs. The password view and the MFA change for fin-04 are thirteen seconds apart, beyond the window, so their order is supported under the stated assumptions. Report the uncertainty value with the timeline so a reader can check each flag.
25.4 The lab (synthetic)
Fictional bundle. The IdP logs in UTC. The booking application logs in local time and its clock, according to its administrator, "may be off by a bit". Save as lab25.py.
# Lab 25 - normalize, estimate offset, join and annotate a synthetic two-source timeline (stdlib only)
import hashlib
from datetime import datetime, timedelta, timezone
from statistics import median
IDP = """2026-03-10T07:12:04Z|signin|fin-04|sess=a91
2026-03-10T07:30:40Z|signin|rcp-02|sess=b17
2026-03-10T07:41:10Z|mfa_change|fin-04|method=phone_added"""
APP = """2026-03-10 04:12:37|session_start|fin-04|sess=a91
2026-03-10 04:31:12|session_start|rcp-02|sess=b17
2026-03-10 04:20:55|export|fin-04|rows=1820
2026-03-10 04:41:30|password_view|fin-04|record=supplier-7"""
for name, raw in (("IDP", IDP), ("APP", APP)):
print(name, "sha256", hashlib.sha256(raw.encode()).hexdigest()[:16])
LOCAL = timezone(timedelta(hours=-3))
def parse(raw, src):
out = []
for line in raw.splitlines():
ts, kind, user, extra = line.split("|")
if src == "IDP":
t = datetime.fromisoformat(ts.replace("Z", "+00:00"))
else:
t = datetime.fromisoformat(ts).replace(tzinfo=LOCAL).astimezone(timezone.utc)
out.append({"src": src, "t": t, "kind": kind, "user": user, "extra": extra, "raw": line})
return out
idp, app = parse(IDP, "IDP"), parse(APP, "APP")
# offset from shared sessions: app session_start vs idp signin with same sess id
pairs = [(a["t"] - i["t"]).total_seconds() for i in idp for a in app
if a["kind"] == "session_start" and i["kind"] == "signin" and a["extra"] == i["extra"]]
offset = median(pairs)
spread = max(pairs) - min(pairs)
print("offset estimates (s):", pairs, "median", offset, "spread", spread)
for a in app:
a["t"] = a["t"] - timedelta(seconds=offset)
events = sorted(idp + app, key=lambda e: e["t"])
unc = spread / 2 + 5 # half the spread plus 5 s assumed processing delay
prev = None
for e in events:
flag = ""
if prev and prev["src"] != e["src"] and (e["t"] - prev["t"]).total_seconds() < unc:
flag = " <- order vs previous uncertain"
print(e["t"].strftime("%H:%M:%S"), e["src"], e["kind"], e["user"], e["extra"], flag)
prev = e
Expected output:
IDP sha256 4eab67acd63e956d
APP sha256 0f7360e825bd7e2c
offset estimates (s): [33.0, 32.0] median 32.5 spread 1.0
07:12:04 IDP signin fin-04 sess=a91
07:12:04 APP session_start fin-04 sess=a91 <- order vs previous uncertain
07:20:22 APP export fin-04 rows=1820
07:30:39 APP session_start rcp-02 sess=b17
07:30:40 IDP signin rcp-02 sess=b17 <- order vs previous uncertain
07:40:57 APP password_view fin-04 record=supplier-7
07:41:10 IDP mfa_change fin-04 method=phone_added
The administrator's remark that the clock "may be off by a bit" is not used anywhere in the code; the offset is estimated from the shared session identifiers a91 and b17 instead. The raw strings are hashed before parsing, so the output can be tied to these exact inputs. Note what the bundle lacks: there is no source other than the application for the export of 1,820 rows, so that event rests on one log.
25.5 Reading the result
The application clock runs about 32.5 seconds fast (after the UTC−3 conversion). Corrected, the sequence for fin-04 is: sign-in 07:12:04; bulk export of 1,820 rows at 07:20:22; viewing a stored supplier password at 07:40:57; adding a phone MFA method at 07:41:10. The last two are 13 seconds apart — larger than the 5.5-second uncertainty used here, so the order (password view before MFA change) is supported under the stated assumptions. One artefact deserves attention: after correction, the rcp-02 application session start (07:30:39) appears one second before the IdP sign-in that created it (07:30:40), which is impossible in reality. It is a residual of using a single median offset when the two pairs disagree by one second (and of printing truncated seconds). The tool correctly flags the pair as order-uncertain; a reader who ignored the flag could invent a story about a session that began before authentication. The fix is not to edit the output but to report the uncertainty, as the flag does.
The printed hash prefixes identify the exact raw inputs; any change to a single character changes them (Chapter 17). Every printed line can be traced to its raw line through the raw field kept in each event.
25.6 Writing the timeline entry
A publishable entry looks like: "07:20:22 UTC (±6 s) — booking application records an export of 1,820 rows by fin-04 (APP line 3; app clock corrected by −32.5 s estimated from two shared sessions)." The bracketed provenance can move to a footnote or appendix, but it must exist.
Write separate entries for events from different sources even when they describe the same action, as with the fin-04 sign-in and session start, and link them rather than merging. A reader can then see two independent records of one moment. For the password view and MFA change, the entry should state the thirteen-second gap and the uncertainty that makes the order supportable.
25.7 Common errors and limits
- Sorting strings from different time zones.
- Applying an administrator's guess ("a bit off") instead of an estimate from shared events.
- Ignoring the residual uncertainty of the offset estimate.
-
Two shared events are a thin basis; more pairs would narrow the uncertainty. If the clock drifts during the day, a single offset is wrong for distant events.
-
Silently correcting the impossible-looking rcp-02 order instead of reporting why it arose.
- Treating the export of 1,820 rows as confirmed by two sources when only the application recorded it.
Exercises — chapter 25
Attempt each exercise before opening the worked results. Every case, organization, person, address and number in an exercise is synthetic unless a cited public source is named.
EX073 — Reproduce and record. Steps: run Lab 25, record the Python version, the two hash prefixes and the corrected time of the export; explain each step of the pipeline in one line. Rubric: 3 = output matches, pipeline steps correctly described; 2 = output only; 1 = not run; 0 = none.
EX074 — Add a source. Inputs: a help-desk ticket "10/03 04:50 user fin-04 says she did not add a phone" (local time, typed by agent). Steps: add it to the timeline with appropriate uncertainty and explain why it cannot be used to order events at second precision. Rubric: 3 = converted to 07:50 UTC, minute precision, entry delay unknown (could be later than the call), marked as statement not system event; 2 = converted without caveats; 1 = wrong zone; 0 = none.
EX075 — Critique the timeline. Steps: list three limits of the Lab 25 timeline that a reviewer should see in the report. Rubric: 3 = offset from two pairs only, possible drift, 5-second processing assumption, single application source for export, local-time assumption for the app; 2 = two; 1 = one; 0 = none.
Chapter synthesis
A defensible timeline preserves and hashes raw inputs, normalizes each source by documented rules, estimates offsets from shared events, flags orderings within uncertainty and points every entry back to its raw record.