◇ OpenCyber Academy
Bachelor's · Chapter 38 · English

Chapter 38 — Dissertation II: findings

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 38 — Dissertation II: findings

Lesson map

Lesson 38 published. Map, example, EX112.

flowchart TD
  N0["Learning outcomes"]
  N1["38.1 Data generation"]
  N2["38.2 Results"]
  N3["38.3 Robustness"]
  N4["38.4 Interpretation"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX112"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["38.1 Data generation"]
  N2["38.2 Results"]
  N3["38.3 Robustness"]
  N4["38.4 Interpretation"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX112"]

[Learning outcomes]
    |
    v
[38.1 Data generation]
    |
    v
[38.2 Results]
    |
    v
[38.3 Robustness]
    |
    v
[38.4 Interpretation]
    |
    v
[Worked example] --> [Exercise EX112]

Learning outcomes

You will (1) carry out the planned analysis; (2) interpret effects with uncertainty; (3) check robustness.

38.1 Data generation

Amara generates synthetic data following her plan. Each store has a baseline invalid entry rate drawn from a range observed in her placement notes, monthly variation and a small general improvement over time. Champion stores receive an additional reduction after month six. She documents every parameter and fixes the random seed. She explains that the known built-in effect lets readers judge whether her method recovers it, which is a test of the method rather than proof about real stores. The parameters appear in an appendix.

38.2 Results

Amara computes invalid entry rates before and after for both groups. Both improve, reflecting the general trend, but champion stores improve more. The difference-in-differences estimate is close to the reduction she built in, and the bootstrap interval excludes zero. She reports the estimate, the interval and the number of stores. She avoids claiming that champions work in reality, stating that the method can detect an effect of this size under these conditions. She presents the results in a table and a chart that show both groups before and after.

38.3 Robustness

She checks robustness. Pre-introduction trends are similar in both groups, supporting the design. Removing the store with the highest baseline does not change conclusions. Repeating the generation with different seeds gives similar estimates. She also tests a scenario where champion stores were already improving faster, and shows that the method would then overstate the effect, illustrating the selection threat she named earlier and explaining why pre-trends must be checked. Amara reports every robustness check in the dissertation, including the one that shows how the method can be misled by different pre-trends.

38.4 Interpretation

Amara interprets findings for practitioners: if champions produce effects similar to those she simulated, a twenty-store comparison could detect them; smaller effects would need more stores or longer periods. She suggests that Harbourline introduce champions in stages to allow a real evaluation. Dr. Lindqvist praises the clear distinction between what the synthetic study shows and what remains unknown, noting that this honesty is essential when studies use synthetic data. Amara writes these recommendations in a separate section addressed to practitioners, so that managers can read them without the technical chapters.

Worked example

Amara generates documented synthetic data, recovers the built-in champion effect with a bootstrap interval excluding zero, checks pre-trends and seeds and explains what the synthetic study can and cannot show.

Synthetic case. Data and results are synthetic.

Errors and limits

Recovering a built-in effect shows the method works, not that the effect is real.

Exercises

Continue studying — lesson 38 (chapter 38 of 40)

← Chapter 37 · Course chapter index · Chapter 39 →

Continue to the next authored lesson.