◇ OpenCyber Academy
Technologist · Chapter 9 · English

Chapter 9 — Evidence and measurement

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 9 — Evidence and measurement

Lesson map

Lesson 09 published. Map, example, EX025.

flowchart TD
  N0["Learning outcomes"]
  N1["9.1 Baselines"]
  N2["9.2 Measuring deviation"]
  N3["Lab — hourly logon baseline"]
  N4["9.3 Reading the result"]
  N5["9.4 Evidence preservation"]
  N6["9.5 Consistent times"]
  N7["9.6 Measurement honesty"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX025"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["9.1 Baselines"]
  N2["9.2 Measuring deviation"]
  N3["Lab — hourly logon baseline"]
  N4["9.3 Reading the result"]
  N5["9.4 Evidence preservation"]
  N6["9.5 Consistent times"]
  N7["9.6 Measurement honesty"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX025"]

[Learning outcomes]
    |
    v
[9.1 Baselines]
    |
    v
[9.2 Measuring deviation]
    |
    v
[Lab — hourly logon baseline]
    |
    v
[9.3 Reading the result]
    |
    v
[9.4 Evidence preservation]
    |
    v
[9.5 Consistent times]
    |
    v
[9.6 Measurement honesty]
    |
    v
[Worked example] --> [Exercise EX025]

Learning outcomes

You will (1) build baselines of normal activity; (2) measure deviations simply and transparently; (3) preserve evidence with integrity; (4) record times and units consistently.

9.1 Baselines

Normal activity varies by hour. Clinic servers are busy during opening hours and almost silent at night. A baseline records typical counts for each hour.

In the lab, the baseline for CLINIC2-SRV covers three hours over ten working days: 01:00, when the clinic is closed, 09:00, when reception starts, and 13:00, after lunch. The night hour holds nine zeros and a single one, while 09:00 ranges from 12 to 16 logons. These two shapes, a near-silent night and a busy morning, are what make an unusual hour stand out at all.

9.2 Measuring deviation

Comparing tonight's count to the baseline's mean and maximum shows whether activity is unusual. Simple rules — "above the highest value seen plus a margin" — are easy to explain to clients.

The lab's rule is exactly of that kind: tonight's 01:00 count is unusual if it exceeds the night maximum of 1 plus a margin of 3, so anything above 4 is flagged. Mei could explain that sentence to the office manager in one line, which would be much harder for a model based on standard deviations alone, even though the lab prints those as well.

Lab — hourly logon baseline

"""Hourly logon baseline for a dental clinic server over 10 working days (synthetic)."""
import statistics
hourly = {1: [0,0,0,1,0,0,0,0,0,0], 9: [14,12,15,13,16,14,12,15,13,14], 13: [9,8,10,9,11,8,9,10,9,8]}
tonight_01 = 7
for h, xs in hourly.items():
    print(f"{h:02d}:00 mean={statistics.mean(xs):4.1f} sd={statistics.pstdev(xs):3.1f} max={max(xs)}")
m, sd = statistics.mean(hourly[1]), statistics.pstdev(hourly[1])
print(f"tonight 01:00 logons={tonight_01} vs mean {m:.1f} -> {'UNUSUAL' if tonight_01 > max(hourly[1]) + 3 else 'normal'}")

Run with python3 lab_baseline.py. Verified output:

01:00 mean= 0.1 sd=0.3 max=1
09:00 mean=13.8 sd=1.2 max=16
13:00 mean= 9.1 sd=0.9 max=11
tonight 01:00 logons=7 vs mean 0.1 -> UNUSUAL

9.3 Reading the result

Over ten working days, the server averaged 0.1 logons at 01:00 (maximum 1), 13.8 at 09:00 and 9.1 at 13:00. Tonight's seven logons at 01:00 are far above the night-time maximum and flagged as unusual.

Mei checks the figures by hand before writing them up. The ten 09:00 values sum to 138, which gives 13.8 as the mean, and the 01:00 values sum to 1, giving 0.1. With a night limit of 4, seven logons are almost twice the limit, so the flag does not depend on a borderline value or a rounding choice. The standard deviations are printed too, but the flag itself uses only the maximum and the margin, which keeps the decision easy to explain.

9.4 Evidence preservation

When an incident starts, analysts export relevant logs, record a hash for each file and note who exported what and when. Exports are stored read-only.

For BrightSmile, Mei exported the security log as E-1 and the firewall log as E-2 and recorded a SHA-256 hash for each. If anyone later questions whether E-1 was changed, recomputing its hash and comparing it to the note settles the question. Read-only storage prevents accidental edits by the analysts who open the exports during the investigation.

9.5 Consistent times

All times are UTC in records, with local time added in client reports. Mixing them causes confusion about sequence.

Mei's incident note records 01:00–02:00 UTC, and the client report adds the clinic's local time beside it. If the note had mixed the server's UTC entries with a local time from a phone call, the 01:36 isolation could appear to come before the 01:22 account creation, and the client would read a sequence that never happened. Analysts at Copperleaf write UTC in every case entry, and only the client-facing summary converts times, which keeps the conversion in one place.

9.6 Measurement honesty

Ten days is a short baseline; holidays and staff changes alter patterns. Mei notes these limits in every report.

A ten-day baseline also covers only working days, so a Saturday clinic session or an early-opening week could produce night or early logons the baseline has never seen. Mei writes the period and the number of days next to every figure, so a reader knows that "maximum 1" means one in ten nights, not one in a year. When a baseline is recalculated, Mei compares the new maximum with the old one, so a sudden jump in normal activity is noticed rather than absorbed.

Clock checks

The agent reports each host's clock offset. A server whose clock drifts several minutes would scramble the sequence of events; Copperleaf alerts when any offset exceeds one minute. During onboarding Mei found one clinic PC eleven minutes fast because it had never synchronised, and corrected it with the contractor.

Updating baselines

Baselines are recalculated monthly from data with confirmed incidents removed, so the attacker's night does not become part of normal.

Worked example

Mei's incident note: "01:00–02:00 UTC: 7 logons to CLINIC2-SRV (baseline mean 0.1, max 1 over 10 days). Exports E-1 (security log) and E-2 (firewall log) saved with SHA-256 hashes at 01:41 by M. Tanaka."

Synthetic case. Counts come from the lab script.

Errors and limits

Baselines learned during an undetected compromise include the attacker's activity as "normal".

Exercises

Synthesis

Baselines make unusual activity visible; preserved, well-timed evidence makes findings trustworthy.

Continue studying — chapter 09 of 30

← Chapter 08 (draft) · Course chapter index · Chapter 10 (draft) →