OpenCyber Academy
Technologist · Chapter 19 · English
Course code: OCA-082Chapter 19: OCA-082-C019

Chapter 19 — Cloud and container security: monitoring

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Lesson map

Lesson 19 published. Map, example, EX055.

flowchart TD
  N0["Learning outcomes"]
  N1["19.1 Audit logs"]
  N2["Lab — audit summary"]
  N3["19.2 Reading the result"]
  N4["19.3 Investigating"]
  N5["19.4 Alerts"]
  N6["19.5 Noise"]
  N7["Retention"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX055"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["19.1 Audit logs"]
  N2["Lab — audit summary"]
  N3["19.2 Reading the result"]
  N4["19.3 Investigating"]
  N5["19.4 Alerts"]
  N6["19.5 Noise"]
  N7["Retention"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX055"]

[Learning outcomes]
    |
    v
[19.1 Audit logs]
    |
    v
[Lab — audit summary]
    |
    v
[19.2 Reading the result]
    |
    v
[19.3 Investigating]
    |
    v
[19.4 Alerts]
    |
    v
[19.5 Noise]
    |
    v
[Retention]
    |
    v
[Worked example] --> [Exercise EX055]

Learning outcomes

You will (1) summarise audit events; (2) identify events needing review; (3) set simple alerts.

19.1 Audit logs

Kubernetes audit logs record requests to the API: who, what action, which resource. With client-b now logging, Priya has data to review. Before client-b's rebuild, its audit logging was off, so there was no record of who had used the old-admin account or what the CI bot read. Each audit event names a user or service account, a verb such as get or create, and a resource such as pods or secrets. The lab summarises synthetic events of that shape for the review period.

Lab — audit summary

"""Summarise synthetic Kubernetes-style audit events and flag ones worth review (stdlib only)."""
import collections
events = [("ci-bot", "create", "deployments"), ("ci-bot", "update", "deployments")] * 20 + \
         [("kwame", "get", "pods")] * 12 + [("priya", "get", "pods")] * 8 + \
         [("ci-bot", "get", "secrets")] * 3 + [("old-admin", "create", "clusterrolebindings"),
          ("old-admin", "get", "secrets"), ("system:anonymous", "get", "pods")]
watch = {"secrets", "clusterrolebindings"}
print(f"events {len(events)}")
for (u, v, r), n in sorted(collections.Counter(events).items()):
    flag = " <- review" if r in watch or u.startswith("system:anon") or u == "old-admin" else ""
    print(f"  {u:17} {v:6} {r:20} {n:3}{flag}")

Run with python3 lab_logs.py. Verified output:

events 66
  ci-bot            create deployments           20
  ci-bot            get    secrets                3 <- review
  ci-bot            update deployments           20
  kwame             get    pods                  12
  old-admin         create clusterrolebindings    1 <- review
  old-admin         get    secrets                1 <- review
  priya             get    pods                   8
  system:anonymous  get    pods                   1 <- review

19.2 Reading the result

Of 66 events, most are routine: the CI bot creating and updating deployments, staff reading pods. Four lines need review: the CI bot reading secrets three times, old-admin creating a cluster role binding and reading a secret, and an anonymous request. The routine lines total 60: 40 deployment events from the CI bot, 12 pod reads by Kwame and 8 by Priya. The six flagged events are spread over four lines, and flagging depends on the resource or the user rather than the volume, which is why a single anonymous request is flagged while twenty deployments are not.

19.3 Investigating

The CI bot reads a secret it needs for deployment, which is expected but its access can be narrowed. The old-admin events predate removal. The anonymous request came before anonymous access was turned off. Each flagged line gets a written conclusion rather than a silent dismissal. The CI bot's three secret reads match deployments that use one stored credential, so Priya proposes limiting its role to that secret. The old-admin events date from before the account was removed, and their timing is examined in the worked example. The anonymous request predates the setting change, and no anonymous request appears after it.

19.4 Alerts

Priya proposes alerts for new cluster role bindings, secret reads by people and any anonymous request. Each alert covers an event that should be rare and always deserves a look. A new cluster role binding changes who can do what across the cluster; a person reading a secret is unusual because workloads, not people, normally use secrets; and any anonymous request after anonymous access was turned off would mean a setting has been reverted. Routine deployment events from the CI bot are summarised, not alerted on.

19.5 Noise

Alerts that fire constantly are ignored; each must lead to an action. If the CI bot's normal deployments raised alerts, the on-call engineer would see dozens a day and soon stop reading them, missing the one that matters. Priya tests each proposed alert against the synthetic events: it should fire for the flagged lines and stay silent for routine ones. Every alert links to a runbook entry stating what to check first and who to tell. An alert that leads to no action for a month is reviewed and either tuned or removed.

Retention

Audit logs are kept for one year in a separate storage account where cluster administrators cannot delete them. Retention lets investigators look back, and separation protects logs from being changed by someone trying to hide activity. Priya checks the storage cost with Kwame: for small clusters, it is a few pounds a month, much less than the value of being able to answer questions after an incident.

She also checks that logs do not contain patient data; audit logs record API requests, not application data, so this is acceptable.

Review rhythm

Flagged events are reviewed each morning by the on-call engineer, with outcomes noted beside each line.

Worked example

Kwame asks whether old-admin's events mean misuse. Priya checks the timestamps: they occurred on the former employee's last day, during a handover. Kwame records this with the client and keeps the log extract.

Synthetic case. Events are synthetic.

Errors and limits

A summary hides detail; investigate flagged lines in full.

Exercises

Synthesis

Summarise logs, investigate flags, and alert only on actionable events.

Continue studying — chapter 19 of 30

← Chapter 18 (draft) · Course chapter index · Chapter 20 (draft) →

Worked answers

EX055. Four: CI bot secret reads, old-admin binding and secret read, anonymous request.

EX056. A handover on the employee's last day.

EX057. New cluster role bindings, secret reads by people, anonymous requests.