◇ OpenCyber Academy
Technologist · Chapter 24 · English

Chapter 24 — Applied research methods

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 24 — Applied research methods

Lesson map

Lesson 24 published. Map, example, EX070.

flowchart TD
  N0["Learning outcomes"]
  N1["24.1 A question from practice"]
  N2["24.2 Method"]
  N3["24.3 Reading research"]
  N4["24.4 Analysis"]
  N5["24.5 Ethics"]
  N6["24.6 Writing up"]
  N7["Choosing method"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX070"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["24.1 A question from practice"]
  N2["24.2 Method"]
  N3["24.3 Reading research"]
  N4["24.4 Analysis"]
  N5["24.5 Ethics"]
  N6["24.6 Writing up"]
  N7["Choosing method"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX070"]

[Learning outcomes]
    |
    v
[24.1 A question from practice]
    |
    v
[24.2 Method]
    |
    v
[24.3 Reading research]
    |
    v
[24.4 Analysis]
    |
    v
[24.5 Ethics]
    |
    v
[24.6 Writing up]
    |
    v
[Choosing method]
    |
    v
[Worked example] --> [Exercise EX070]

Learning outcomes

You will (1) ask a focused practical question; (2) collect and analyse data carefully; (3) read relevant research.

24.1 A question from practice

Priya asks: how often do Cobalt's clusters drift from baseline between checks? It is answerable with data Cobalt already has. The question comes from the baseline work: the guided lab showed settings at one moment, but it could not show whether clusters stayed compliant between runs. If drift is frequent, daily checks matter more than the initial cleanup; if it is rare, a weekly check might be enough. Either answer would change how Cobalt schedules its baseline checks. Priya writes the question down before collecting any data, as her measurement plan requires.

24.2 Method

She runs the baseline check daily for four weeks on synthetic copies of the settings history and counts new failures. A drift event is counted when a setting that passed the previous day fails on the next run. Using synthetic copies of the settings history means the study needs no new access to client clusters. Running the same six checks every day for four weeks gives a consistent measure, and each new failure is linked to the change record that caused it, where one exists.

24.3 Reading research

Armbrust et al. (2010) describe the cloud computing model and its obstacles, including configuration complexity. Verma et al. (2015) describe Borg, Google's cluster manager, explaining how large clusters are managed by automation, which supports Cobalt's move to automated checks. Burns et al. (2016) add the lessons Google drew from Borg and Omega when building Kubernetes, including the value of declaring desired state and letting automation reconcile it. None of these papers studies drift at a small hosting company, so Priya uses them for context rather than as evidence for her own finding.

24.4 Analysis

She counts drift events per cluster and notes causes, such as manual console changes. For each drift event she records the cluster, the setting, the day and the cause found in change records, such as a manual console change during an urgent fix. Counting causes as well as events matters, because the recommendation depends on why settings drift, not only how often. Where no change record exists, she records the cause as unknown rather than guessing. She then compares the causes across the three clusters to see whether any pattern repeats.

24.5 Ethics

Data describes systems, not individual performance; names are removed from change records. The study is about how clusters drift, not about which engineer made a change, so removing names keeps the focus on processes. Without that step, a report showing that most drift came from urgent manual fixes could be read as blaming the people on call. Priya confirms the approach with Kwame before starting, and states it in the report. Change records with names are read only inside the placement workspace and never copied into the report.

24.6 Writing up

A two-page report states question, method, results and limits. The report states the question in one sentence, describes the four weeks of daily checks on three clusters, presents drift counts and causes, and lists the limits, including the short period and the 84 cluster-days. The recommendation follows from the results: urgent fixes should be recorded in the pipeline afterwards so the baseline catches up. Kwame reads the draft first and asks for the limits to appear before the recommendation, which Priya accepts. The final report keeps the same structure as her placement plan, so readers can find the evidence for each claim.

Choosing method

A survey of staff might have shown opinions about drift; daily checks show actual drift. Priya chooses the method that answers her question with the least guesswork. She also considers sample size: three clusters over four weeks gives 84 cluster-days, enough to see patterns but not to compare clusters precisely.

She writes these limitations into the report before showing results, so readers interpret the findings with care.

Next study

She suggests repeating the study after the nightly comparison is introduced to see if drift falls.

Sharing findings

Priya presents the study to the team in fifteen minutes, focusing on the recommendation and its evidence.

Data handling

The settings history copies were stored only in the placement workspace and deleted after the report.

Worked example

The study finds most drift came from manual changes during urgent fixes. Priya recommends recording urgent fixes in the pipeline afterwards so the baseline catches up.

Synthetic case. Study data is synthetic.

Errors and limits

Four weeks is short; seasonal work might differ.

Exercises

Synthesis

Small focused studies can guide practical decisions.

Continue studying — chapter 24 of 30

← Chapter 23 (draft) · Course chapter index · Chapter 25 (draft) →