◇ OpenCyber Academy
Engineering · Chapter 38 · English

Chapter 38 — Cloud and container security: validation

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 38 — Cloud and container security: validation

Lesson map

Lesson 38 published. Map, example, EX112.

flowchart TD
  N0["Learning outcomes"]
  N1["38.1 Verification versus validation"]
  N2["38.2 Goals review"]
  N3["38.3 Developer validation"]
  N4["38.4 Auditor validation"]
  N5["38.5 Incident validation"]
  N6["38.6 Adjustments"]
  N7["Acting on validation"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX112"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["38.1 Verification versus validation"]
  N2["38.2 Goals review"]
  N3["38.3 Developer validation"]
  N4["38.4 Auditor validation"]
  N5["38.5 Incident validation"]
  N6["38.6 Adjustments"]
  N7["Acting on validation"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX112"]

[Learning outcomes]
    |
    v
[38.1 Verification versus validation]
    |
    v
[38.2 Goals review]
    |
    v
[38.3 Developer validation]
    |
    v
[38.4 Auditor validation]
    |
    v
[38.5 Incident validation]
    |
    v
[38.6 Adjustments]
    |
    v
[Acting on validation]
    |
    v
[Worked example] --> [Exercise EX112]

Learning outcomes

You will (1) validate that the engineered system meets stakeholder needs, not just requirements; (2) gather evidence from users, auditors and incidents; (3) compare with goals.

38.1 Verification versus validation

Verification asks "did we build it right?" (requirements met). Validation asks "did we build the right thing?" (stakeholders' needs met). A system could pass every test yet frustrate developers so much that they bypass it.

Verification uses the requirements and tests already written: admission rejects unsigned images, workload identities hold no long-lived keys. Validation needs evidence from people: developers who deploy every day, auditors who rely on the records, and the incident history. Both can fail independently. A platform that meets every requirement but takes a week to grant an exception will push teams towards workarounds, and those workarounds are where the next incident is likely to come from.

38.2 Goals review

Elena compares results with chapter 20's goals: no long-lived workload keys (achieved), default-deny in all production namespaces (achieved), Restricted profile with documented exceptions (achieved, one exception), signed images only (achieved), detection within objectives (achieved after alert fixes).

Each goal is marked with the evidence behind it: the key inventory, the namespace policy report, the exception register, the admission logs and the detection test results. The one Restricted-profile exception is listed with its owner and end date, so 'achieved' does not hide it. Detection was only met after alert fixes, and Elena says so, because a goal reached late tells reviewers where the original design was weak.

38.3 Developer validation

A survey of 30 developers asks about deployment friction and clarity of rejections. 70% find rejection messages clear; complaints focus on exception request time.

The survey is anonymous and short, with questions about specific moments, such as the last rejected deployment, rather than general opinions about security. Seventy percent clear also means about nine of the thirty found messages unclear, so Elena reads their comments and finds rejections that name a policy without saying how to fix the manifest. Complaints about exception time match the review data, which shows requests waiting several days for an answer.

38.4 Auditor validation

An internal auditor reviews evidence against CSA CCM v4 mappings and finds evidence sufficient for identity and infrastructure domains, asking for more on key rotation records.

The auditor works from the mappings to the matrix's controls, sampling evidence for each and checking that it is dated, complete and produced by the system rather than written up by hand afterwards. Rotation was the weak area because keys were rotated on schedule but the records were scattered across tickets and chat messages. The request is reasonable: a control nobody can show happened is hard to rely on in an audit.

38.5 Incident validation

Six months without a credential or exposure incident is encouraging but short; Elena reports it with that caveat.

Six months is a short window for events that may be rare in any case, so a quiet period could simply mean nobody tried. Elena adds the evidence that the controls were exercised: blocked attempts such as the public sandbox bucket, rejected unsigned images and alerts confirmed in game days. Together these show the controls acting, which says more than the absence of incidents alone, and she states which of them came from tests rather than real events.

38.6 Adjustments

Exception requests get a 2-day service level; key rotation records are generated automatically.

Each adjustment answers a specific validation finding. The service level addresses the developers' main complaint and is tracked as a metric, so slow responses become visible. Automatic records answer the auditor: every rotation now writes an entry with the key identifier, the time and the identity that performed it. The two changes are small, which is typical. Validation more often reveals friction and gaps in evidence than flaws in the controls themselves.

Acting on validation

Validation results become backlog items with owners. The exception service level and automatic rotation records are delivered within a month, and the next survey shows exception complaints falling. Elena repeats validation every six months, because needs change as the platform and teams grow. She reports validation alongside verification so leadership sees both whether controls work and whether people can live with them.

Worked example

The survey also reveals that developers no longer remember why the network policies exist. Elena adds short "why" notes to each policy file linking to the opening case and the threat analysis, so the reasons survive staff changes.

Synthetic case. Survey and audit results are synthetic.

Errors and limits

Absence of incidents in a short period is weak evidence of effectiveness.

Exercises

Synthesis

Validate with the people who use and audit the system.

Continue studying — chapter 38 of 50

← Chapter 37 (draft) · Course chapter index · Chapter 39 (draft) →