◇ OpenCyber Academy
Engineering · Chapter 8 · English

Chapter 8 — Computer architecture

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 8 — Computer architecture

Lesson map

Lesson 08 published. Map, example, EX022.

flowchart TD
  N0["Learning outcomes"]
  N1["8.1 The hierarchy"]
  N2["8.2 Bound by what"]
  N3["8.3 Caching enrichment"]
  N4["8.4 Hardware roots of trust"]
  N5["8.5 Memory sizing"]
  N6["Practice note — Profiling discipline"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX022"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["8.1 The hierarchy"]
  N2["8.2 Bound by what"]
  N3["8.3 Caching enrichment"]
  N4["8.4 Hardware roots of trust"]
  N5["8.5 Memory sizing"]
  N6["Practice note — Profiling discipline"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX022"]

[Learning outcomes]
    |
    v
[8.1 The hierarchy]
    |
    v
[8.2 Bound by what]
    |
    v
[8.3 Caching enrichment]
    |
    v
[8.4 Hardware roots of trust]
    |
    v
[8.5 Memory sizing]
    |
    v
[Practice note — Profiling discipline]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX022]

Learning outcomes

You will (1) relate CPU, memory and storage hierarchy to intelligence processing; (2) estimate whether a workload is CPU-, memory- or I/O-bound; (3) explain caching for enrichment lookups; (4) recognise hardware trust anchors in collection systems.

8.1 The hierarchy

Registers and caches are fast and small; main memory larger and slower; solid-state storage slower again; network storage slowest. Lantern's hot set — indicators matched against live traffic — must fit in memory, or every match waits on disk.

Kofi checked this with a simple test. Matching a batch against an in-memory set took a few milliseconds; the same lookups against an on-disk index took tens of milliseconds, mostly waiting for reads. The cold archive can live on slower storage because analysts query it occasionally and can wait a second for an answer.

8.2 Bound by what

Parsing JSON-formatted STIX objects is CPU-heavy. Joining indicators with sightings is memory-heavy. Loading archives is I/O-heavy. Profiling identifies the binding resource; buying the wrong hardware is common and expensive.

Kofi learned this when a request for faster disks arrived to fix slow parsing. Profiling showed the parser was using a full processor core while the disks sat idle. Faster disks would have changed nothing. A second worker process, which cost nothing but configuration, roughly doubled throughput. Each profile is saved with the hardware it ran on, so later comparisons are fair.

8.3 Caching enrichment

Enrichment — adding WHOIS-style registration data, passive DNS or internal asset context — often repeats lookups. A cache keyed by indicator with a time-to-live avoids repeated queries and respects third-party rate limits and terms. Cached data must expire, both for accuracy and because some providers' terms limit retention.

Kofi sets the time to live per provider, using the shortest of what accuracy needs and what the provider's terms allow. The cache records when each entry was fetched, so an analyst can tell whether data is fresh.

8.4 Hardware roots of trust

Collection servers that sign exported data rely on keys. A hardware security module or TPM keeps signing keys out of ordinary memory, improving the integrity of what Lantern shares.

Without such a module the key sits in a file that anyone with administrator access to the server could copy, and a copied key lets an attacker sign false indicators that partners would trust. With it, signing still works but the key cannot be exported. Kofi's design lists which servers hold signing keys and who may use them, and the module's logs show every signing operation for later review.

8.5 Memory sizing

One million indicators averaging 200 bytes in an in-memory set need about 200 MB plus overhead; Python objects can triple that. Measure.

Kofi built a set of a million test strings and measured it rather than trusting the arithmetic. The result was several times the raw data size, because each string object carries its own overhead. He then sized the matcher's memory with room for a year of growth and a margin, and recorded the measurement method so it can be repeated when volumes change. A compact representation, such as storing hashes of values, could reduce the footprint if memory became tight.

Practice note — Profiling discipline

Kofi profiles before and after every performance change and keeps the reports. Changes that do not move the measured bottleneck are rolled back even if they 'should' help. This habit stops the team from accumulating clever optimisations that add complexity but no speed.

Worked example

Profiling shows Lantern's matcher spends 70% of its time waiting on disk. Kofi moves the active indicator set (about 400,000 entries) into memory, cutting match latency from 40 ms to 3 ms per batch, and adds a 24-hour TTL cache for enrichment lookups to stay within a provider's daily query quota.

Synthetic case. Profiles and timings are invented.

Errors and limits

Caches can serve stale data; TTLs are a correctness choice, not only a performance one. Respect provider terms on retention.

Exercises

Synthesis

Know which resource binds, keep the hot set in memory, cache with expiry, and anchor signing keys in hardware.

Continue studying — chapter 08 of 50

← Chapter 07 (draft) · Course chapter index · Chapter 09 (draft) →