◇ OpenCyber Academy
Engineering · Chapter 42 · English

Chapter 42 — Cloud and container security: applied research

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 42 — Cloud and container security: applied research

Lesson map

Lesson 42 published. Map, example, EX124.

flowchart TD
  N0["Learning outcomes"]
  N1["42.1 The question"]
  N2["42.2 Data"]
  N3["42.3 Analysis"]
  N4["42.5 Ethics"]
  N5["42.6 Reporting"]
  N6["Data quality"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX124"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["42.1 The question"]
  N2["42.2 Data"]
  N3["42.3 Analysis"]
  N4["42.5 Ethics"]
  N5["42.6 Reporting"]
  N6["Data quality"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX124"]

[Learning outcomes]
    |
    v
[42.1 The question]
    |
    v
[42.2 Data]
    |
    v
[42.3 Analysis]
    |
    v
[42.5 Ethics]
    |
    v
[42.6 Reporting]
    |
    v
[Data quality]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX124]

Learning outcomes

You will (1) run a small applied study on the platform; (2) frame a question; (3) collect and analyse data; (4) report limits.

42.1 The question

Elena asks: "Since admission enforcement began, has the time from a risky workload configuration being proposed to being fixed fallen?" Before enforcement, risky settings were found by weekly scans after deployment.

The question is narrow enough to answer with data Solent already holds, and it matters to developers as well as to security: a risky setting fixed within hours costs less than one found in production a week later. Elena writes the question, the measure and the comparison periods down before she looks at any numbers. Fixing them first stops her from choosing whichever measure happens to show the largest improvement once the results are in.

42.2 Data

Pipeline and admission logs give the time a risky configuration first appeared in a pull request or admission request, and when a compliant version replaced it. Six months before and after are compared.

For the period before enforcement, the start time comes from the pull request that introduced the setting and the end from the change that fixed it, found through the weekly scan records. Afterwards, the start is the rejected admission request and the end is the first accepted version of the same workload. Linking the two events needs a stable workload name, so records where a service was renamed between them are set aside and counted in the report.

42.3 Analysis

Median time to fix falls from 9 days (found after deployment) to 4 hours (rejected at admission, fixed before deployment). Most fixes now happen before anything runs.

Medians are used because a few very slow fixes would distort an average. Elena also reports the spread, since some fixes after enforcement still took days, mostly for workloads that needed an exception rather than a manifest change. Comparing nine days with four hours overstates the control's own effect, because part of the old figure was simply waiting for the weekly scan. She notes this in the results rather than leaving it for the limits.

42.4 Limits

The before period relied on weekly scans, so measured times include detection delay; teams also improved through training; one organisation.

Each limit points in a known direction. Detection delay and training both make the improvement look larger than enforcement alone would produce. A single organisation means the result may not hold where teams deploy less often or use other tools. Elena does not try to remove these effects with statistics the data cannot support; she names them, estimates the scan delay at up to a week, and states that the finding is a strong local improvement, not a general law.

42.5 Ethics

Data are aggregated by namespace; no individual developer is identified.

Individual-level data could be used to judge or rank developers, which was never the study's purpose and would make teams wary of the admission controls themselves. Aggregating by namespace answers the question without that risk. Elena also confirms with Rosa that using pipeline logs for an internal study fits the purpose they are kept for, and the study's script reads only the fields it needs: timestamps, namespace, workload name and the policy that was triggered.

42.6 Reporting

A four-page report with method, results, limits and recommendations is shared internally and summarised for the regulator.

The internal report keeps the method detailed enough to be repeated next year with the same script. The regulator's summary is one page: what was measured, the main result, the limits and what Solent is changing because of it, which is the secure service template. Both versions state the limits in the same words, so the shorter one does not sound more certain than the evidence. Elena files the data and script with the report.

Data quality

Before analysis, Elena checks that pipeline and admission timestamps use the same clock and time zone, and removes 11 records from a test namespace. She keeps the cleaning steps in a script with the analysis, so the results can be reproduced. A second engineer reruns the script and gets the same medians, which goes into the report as a reproducibility check. Elena also lists the fields she could not use, such as fix effort, which would make a useful follow-up study.

Worked example

The study's most useful finding is unexpected: 40% of admission rejections occur in the first week of a new service. Solent adds a secure service template so new services start compliant, which reduces first-week rejections by half in the following quarter.

Synthetic case. Data and results are synthetic.

Errors and limits

Before-after comparisons cannot isolate one cause; describe other changes.

Exercises

Synthesis

Applied studies show what controls change and where to improve next.

Continue studying — chapter 42 of 50

← Chapter 41 (draft) · Course chapter index · Chapter 43 (draft) →