◇ OpenCyber Academy
Engineering · Chapter 6 · English

Chapter 6 — Calculus II

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 6 — Calculus II

Lesson map

Lesson 06 published. Map, example, EX016.

flowchart TD
  N0["Learning outcomes"]
  N1["6.1 Integrals as totals"]
  N2["6.2 Exponential growth"]
  N3["6.3 Decay"]
  N4["6.4 Approximations"]
  N5["6.5 Case"]
  N6["6.6 Retention rules"]
  N7["Rates of change in cost"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX016"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["6.1 Integrals as totals"]
  N2["6.2 Exponential growth"]
  N3["6.3 Decay"]
  N4["6.4 Approximations"]
  N5["6.5 Case"]
  N6["6.6 Retention rules"]
  N7["Rates of change in cost"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX016"]

[Learning outcomes]
    |
    v
[6.1 Integrals as totals]
    |
    v
[6.2 Exponential growth]
    |
    v
[6.3 Decay]
    |
    v
[6.4 Approximations]
    |
    v
[6.5 Case]
    |
    v
[6.6 Retention rules]
    |
    v
[Rates of change in cost]
    |
    v
[Worked example] --> [Exercise EX016]

Learning outcomes

You will (1) use integrals to total quantities over time; (2) apply exponential growth and decay to log volumes and retention; (3) use series approximations for capacity planning; (4) connect continuous models to storage costs.

6.1 Integrals as totals

Storage needed for logs is the integral of ingestion rate over the retention period. If a SIEM ingests r(t) gigabytes per day, storage after T days is the area under r(t) from 0 to T.

When the rate is constant, the area is just a rectangle: rate times days. Real ingestion rises on weekdays, falls at weekends and jumps during incidents, so the area has an uneven top edge. Summing daily figures from the SIEM's own volume reports gives the area directly, and comparing that sum with the rectangle estimate shows how much the simple model misses.

6.2 Exponential growth

If log volume grows 3% per month, volume after n months is V₀·1.03ⁿ. After two years it is about double. Capacity plans that assume flat volume fail.

The doubling comes from 1.03²⁴ ≈ 2.03. A useful shortcut is the rule of 70: dividing 70 by the percentage growth rate gives the approximate doubling time, here about 23 months. Growth like this comes from new systems being connected, more verbose logging after incidents and business expansion, so it tends to continue unless someone deliberately trims sources. Amara plots both projections.

6.3 Decay

Value of old logs decays: most investigations use recent data. A simple model treats the chance of needing a day's logs as decaying exponentially with age, informing tiered storage — hot, warm, cold.

Under such a model, if the chance of needing a given day halves every month, logs three months old are needed an eighth as often as fresh ones. That suggests fast, expensive storage for recent data and slower, cheaper storage later. The model is a simplification: some investigations, such as slow insider cases, reach far back, which is why cold storage is kept searchable rather than discarded.

6.4 Approximations

For small growth rates, (1 + g)ⁿ ≈ 1 + ng for short periods, a quick check on spreadsheets.

At 3% over three months, the linear estimate gives 9% against an exact 9.27%, close enough for a sanity check. Over two years it gives 72% against an exact 103%, badly wrong. The approximation's value is in catching gross spreadsheet errors quickly, such as a misplaced decimal, not in replacing the exact calculation for planning. Amara uses it to check her storage spreadsheet: a 3% monthly rise over one quarter should add roughly 9% to ingestion, and a cell showing 90% would point straight to a misplaced decimal.

6.5 Case

Lindqvist's SIEM ingests 120 GB per day and keeps 90 days hot. Paolo asks Amara whether the current 12 TB hot tier will last a year if volume grows 3% monthly.

The question has two parts: how much hot storage today's rate needs, and how that need changes as ingestion grows over the year. Amara sets it up as a rate multiplied by the retention window, first at today's rate and then at the rate a year from now. She notes that the hot tier holds a rolling window, so what matters is the ingestion rate at the time, not the total collected during the year.

6.6 Retention rules

Retention is also set by policy and law; NIST SP 800-92 recommends defining retention per log type. Maths informs cost; policy decides minimums.

A log type needed for regulatory audits may have to be kept for years, even if investigations rarely touch it; debug logs from a test system may be kept for days. The SOC proposes retention based on investigative value, legal and compliance teams set minimums, and storage cost shapes which tier each type sits in for each part of its life.

Rates of change in cost

The derivative of storage cost with respect to retention days is the daily ingestion cost. Seeing cost as a rate helps managers compare keeping ten more days of logs against other uses of the budget. Amara presents both the total and the marginal cost for each log type, so decisions can be made source by source rather than all at once.

Worked example

Amara computes today's hot need: 120 GB × 90 = 10.8 TB. In 12 months ingestion is 120 × 1.03¹² ≈ 171 GB/day, so 90 days need about 15.4 TB, exceeding 12 TB. She recommends moving firewall logs older than 30 days to the warm tier.

Synthetic case. Volumes and tiers are invented.

Errors and limits

Growth rarely follows a smooth curve; new log sources add steps. Revisit plans quarterly.

Exercises

Synthesis

Integrals and growth models turn ingestion rates into storage plans, within retention limits set by policy.

Continue studying — chapter 06 of 50

← Chapter 05 (draft) · Course chapter index · Chapter 07 (draft) →