OpenCyber Academy
Master's · Chapter 6 · English
Course code: OCA-179Chapter 06: OCA-179-C006

Chapter 6 — Data collection and analysis

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 06 published. Map, example, EX036.

flowchart TD
  N0["Learning outcomes"]
  N1["6.1 Validate first"]
  N2["Lab 2 — STIX property check"]
  N3["6.2 What the check shows"]
  N4["Lab 3 — time-to-block medians"]
  N5["6.3 Reading the medians"]
  N6["6.4 Description versus cause"]
  N7["6.5 Reporting data quality back"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX036"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["6.1 Validate first"]
  N2["Lab 2 — STIX property check"]
  N3["6.2 What the check shows"]
  N4["Lab 3 — time-to-block medians"]
  N5["6.3 Reading the medians"]
  N6["6.4 Description versus cause"]
  N7["6.5 Reporting data quality back"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX036"]

[Learning outcomes]
    |
    v
[6.1 Validate first]
    |
    v
[Lab 2 — STIX property check]
    |
    v
[6.2 What the check shows]
    |
    v
[Lab 3 — time-to-block medians]
    |
    v
[6.3 Reading the medians]
    |
    v
[6.4 Description versus cause]
    |
    v
[6.5 Reporting data quality back]
    |
    v
[Worked example] --> [Exercise EX036]

Learning outcomes

You will (1) validate incoming structured data before analysis; (2) compute and compare medians for skewed timing data; (3) report effect sizes with honest uncertainty; (4) separate descriptive findings from causal claims.

6.1 Validate first

Members export STIX 2.1 objects from CPSG's TAXII collection with their block logs. Before any statistics, Lucía checks that objects carry required properties and well-formed identifiers. Malformed objects are counted and reported to the community, not silently repaired.

Lucía's demonstration batch checks three objects, and only one passes. Across the full study export, 94% of indicators were valid. Counting failures before analysis tells the reader how much of CPSG's data could contribute timings at all. An invalid object that a member still blocked or monitored is counted under Aim 3, because the member's decision happened even though its publication time cannot be trusted.

Lab 2 — STIX property check

"""Lab 2 (ch 06): check synthetic STIX 2.1-style objects for common required properties."""
import json, re
objs = json.loads("""[
 {"type":"indicator","spec_version":"2.1","id":"indicator--8e2e2d2b-17d4-4cbf-938f-98ee46b3cd3f",
  "created":"2026-09-01T10:00:00.000Z","modified":"2026-09-01T10:00:00.000Z",
  "pattern":"[domain-name:value = 'c2.example']","pattern_type":"stix","valid_from":"2026-09-01T10:00:00Z"},
 {"type":"indicator","spec_version":"2.1","id":"indicator-123",
  "created":"2026-09-02T10:00:00.000Z","modified":"2026-09-02T10:00:00.000Z","pattern_type":"stix"},
 {"type":"threat-actor","id":"threat-actor--0f9c3a5e-6b1d-4c39-9a8e-2b9f6a7d1c11",
  "created":"2026-09-03T10:00:00.000Z","modified":"2026-09-03T10:00:00.000Z","name":"SYNTH-CARRION"}
]""")
COMMON = ["type", "spec_version", "id", "created", "modified"]
EXTRA = {"indicator": ["pattern", "pattern_type", "valid_from"], "threat-actor": ["name"]}
for o in objs:
    miss = [k for k in COMMON + EXTRA.get(o["type"], []) if k not in o]
    bad_id = not re.fullmatch(o["type"] + r"--[0-9a-f-]{36}", o.get("id", ""))
    probs = miss + (["id format"] if bad_id else [])
    print(o.get("id"), "OK" if not probs else "PROBLEMS: " + ", ".join(probs))

Run with python3 lab_stix.py. Verified output:

indicator--8e2e2d2b-17d4-4cbf-938f-98ee46b3cd3f OK
indicator-123 PROBLEMS: pattern, valid_from, id format
threat-actor--0f9c3a5e-6b1d-4c39-9a8e-2b9f6a7d1c11 PROBLEMS: spec_version

6.2 What the check shows

The first indicator passes. The second lacks pattern and valid_from and has an identifier that is not type--UUID. The threat-actor object lacks spec_version, which STIX 2.1 requires on domain objects. Only valid objects contribute t_pub values.

Each failure has a different consequence. Without valid_from, the indicator has no trustworthy publication time, so it cannot contribute to time-to-block. A malformed identifier breaks links between objects. A missing spec_version suggests the publishing tool predates STIX 2.1 or is misconfigured, which is useful feedback for that member class.

Lab 3 — time-to-block medians

"""Lab 3 (ch 06): median time-to-block (hours) for synthetic sharing-community members vs non-members."""
from statistics import median
members = [2.0, 3.5, 1.0, 6.0, 2.5, 4.0, 3.0]
non_members = [9.0, 14.0, 5.0, 22.0, 11.0, 8.0]
mm, mn = median(members), median(non_members)
print(f"members n={len(members)} median={mm:.1f}h")
print(f"non-members n={len(non_members)} median={mn:.1f}h")
print(f"difference={mn - mm:.1f}h ratio={mn / mm:.2f}")

Run with python3 lab_ttb.py. Verified output:

members n=7 median=3.0h
non-members n=6 median=10.0h
difference=7.0h ratio=3.33

6.3 Reading the medians

Timing data are skewed — a few slow cases dominate means — so medians are reported. Members' median is 3.0 hours, non-members' 10.0 hours: a difference of 7.0 hours, ratio 3.33. With samples of seven and six, uncertainty is wide; a bootstrap interval or a rank-based test would be reported beside the medians.

Lucía's write-up puts the 3.0- and 10.0-hour medians next to their interval, not alone in a sentence. With six non-members, one unusually slow port can move the median by hours, so the interval tells members how much weight the difference can bear.

6.4 Description versus cause

The result describes faster blocking among members in this sample. Given the confounders discussed in Chapter 3, Lucía writes "is associated with" rather than "causes", unless the before-and-after stratum shows a consistent change on joining.

The four ports that joined CPSG in 2025 are the stratum that could support a stronger claim. If each blocked faster after joining than before, under the same staff and tools, the association becomes harder to explain by size or budget, though still not proof of cause.

6.5 Reporting data quality back

The validation counts are themselves a contribution. Lucía sends CPSG a short data-quality note listing how many objects lacked required properties, grouped by publishing member class, so producers can fix their exports. The note follows the same aggregation rule as the dissertation.

With 94% of indicators valid in the full export, the note can tell producers that one object in roughly seventeen could not contribute a publication time. Grouping by member class, not by member, keeps the note within the board's aggregation rule.

Worked example

Lucía reports: 3 objects checked, 1 valid in the demonstration batch; across the real study export (synthetic), 94% of indicators were valid. Median time-to-block 3.0 h for members versus 10.0 h for matched non-members, with small samples flagged.

Synthetic case. All objects, timings and percentages are invented.

Errors and limits

Repairing malformed data by guessing creates fiction. Reporting a ratio without sample sizes misleads readers.

Exercises

Synthesis

Validate before counting, prefer medians for skewed times, show sample sizes and uncertainty, and say "associated with" until design supports more.

Continue studying — chapter 06 of 08

← Chapter 05 (draft) · Course chapter index · Chapter 07 (draft) →

Worked answers

EX036. PROBLEMS: pattern, valid_from, id format.

EX037. It lacks spec_version, which STIX 2.1 requires on every domain object.

EX038. A few very slow blocks skew the distribution; the median reflects the typical case without being dragged by outliers.

EX039. Members median 3.0 h, non-members median 10.0 h, ratio 3.33.

EX040. Example: "In this small sample (7 members, 6 non-members), membership was associated with a lower median time-to-block (3.0 h vs 10.0 h); causation is not established."

EX041. A bootstrap confidence interval for the difference in medians, resampling each group with replacement many times; or a Mann–Whitney rank test.

EX042. No. Exclude the object from timing analysis and report it as a data-quality finding; inventing values fabricates data.