Chapter 8 — Independent review and defense
Lesson map
Lesson 08 published. Map, example, EX045.
flowchart TD
N0["Learning outcomes"]
N1["8.1 Independent review"]
N2["8.2 Defense questions"]
N3["8.3 Answers"]
N4["8.4 Follow-up"]
N5["8.5 Reflection"]
N6["What the examiners valued"]
N7["Course review"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX045"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["8.1 Independent review"]
N2["8.2 Defense questions"]
N3["8.3 Answers"]
N4["8.4 Follow-up"]
N5["8.5 Reflection"]
N6["What the examiners valued"]
N7["Course review"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX045"]
[Learning outcomes]
|
v
[8.1 Independent review]
|
v
[8.2 Defense questions]
|
v
[8.3 Answers]
|
v
[8.4 Follow-up]
|
v
[8.5 Reflection]
|
v
[What the examiners valued]
|
v
[Course review]
|
v
[Worked example] --> [Exercise EX045]
Learning outcomes
You will (1) undergo independent review; (2) defend findings; (3) plan follow-up.
8.1 Independent review
A second examiner re-runs the analysis from the code and extract and obtains the same tables.
Re-running from the extract tests more than arithmetic: it checks that the pseudonymised extract, the coding rules for missing fields and the scripts together produce the dissertation's tables. A difference would point to an undocumented manual step. The second examiner's identical tables also confirm that the risk ratio equal to 3.67 and the other headline figures were not mistyped. His tables, including 66 of 120 admin laptops drifted, match the dissertation figure for figure.
8.2 Defense questions
Examiners ask about the remote result, missing data and generalisation.
Each question targets a choice that could change the conclusion. The remote result tests whether Chloé understood the overlap with admin rights. Missing data tests whether the no-refresh coding biased Explanation B's test. Generalisation tests whether she claims more than one manufacturer's fleet can show. Chloé prepared each answer from the dissertation's own tables, not from memory. All three questions concern results that depend on design choices, the stratification, the no-refresh coding and the single partner, rather than on arithmetic.
8.3 Answers
Chloé explains the stratified comparison, the no-refresh coding and that results apply to centrally managed fleets similar to Halvorsen.
Each answer points to a decision made before the data: the stratification was in the proposal, the no-refresh rule came from the pilot, and the generalisation statement matches the limitations chapter. Answers anchored in earlier decisions show the examiners that the conclusions were not shaped after seeing the results. She did not claim that the findings apply to unmanaged or personally owned devices, where the baseline is not enforced in the same way.
8.4 Follow-up
Halvorsen removes standing admin rights from 120 devices; a follow-up six months later will test whether drift falls.
The 120 devices are exactly the admin group in Chloé's data. The follow-up will compare their drift rate after six months with the 55.0% before, using the same monthly checks and coding rules. Halvorsen's same-day elevation process, which reduced complaints, is recorded too, so that a change in drift can be read alongside how often temporary elevation was used. Using the same coding rules for missing fields means the before and after drift rates are directly comparable.
8.5 Reflection
Pre-stated predictions kept the analysis focused.
Writing them before seeing the data meant the engineering result could not be reinterpreted afterwards as a weak effect worth pursuing, and the remote result could not be presented without the stratification that explains it. Chloé notes that the same approach, stated predictions with dated entries in the plan, is what she would use in the follow-up study. Without them, three comparisons with different strengths would have invited a story chosen after the fact; with them, each result had a place to go before it was known, and the examiners could check that it went there.
What the examiners valued
Examiners praise the pre-registered predictions, the stratified analysis and the honest treatment of missing data. They suggest that a follow-up could measure drift weekly rather than monthly to capture short changes, and that interviewing users could explain why admin rights were requested in the first place.
Chloé incorporates these suggestions into her future work section.
Course review
The course moved from a practical question through literature, baselines, design and ethics, competing explanations, pilot, analysis, writing and defense, each step building on the last.
Sharing with Halvorsen
Chloé presents a short summary to Halvorsen's IT team, focusing on the admin finding, the finance installer and the re-measurement plan.
Data deletion
After examination, the extract is deleted as agreed, and the deletion is confirmed in writing to Halvorsen.
Thanks
She thanks Dr Mehta, Sven and the IT team.
Publication
With Dr Mehta, Chloé plans a short practitioner article summarising the admin finding for IT managers, using only aggregate figures.
Next steps
Chloé hopes to run the follow-up study with Halvorsen and compare drift before and after admin rights were removed.
Worked example
An examiner asks whether users resisted. Chloé reports that Halvorsen added a same-day elevation process, which reduced complaints.
Synthetic case. Synthetic.
Errors and limits
A defense is stronger when weaknesses are named before examiners raise them.
Exercises
- EX045 — Reproduction. How was the analysis independently checked? Rubric: 3 = examiner re-ran code; 2 = partial; 1 = vague; 0 = none.
- EX046 — Questions. Which topics did examiners raise? Rubric: 3 = remote result, missing data, generalisation; 2 = two; 1 = one; 0 = none.
- EX047 — Scope of findings. To which organisations do findings apply? Rubric: 3 = centrally managed similar fleets; 2 = partial; 1 = vague; 0 = none.
- EX048 — Follow-up. What follow-up is planned? Rubric: 3 = remove admin from 120, re-measure in six months; 2 = partial; 1 = vague; 0 = none.
- EX049 — Resistance. How was user resistance handled? Rubric: 3 = same-day elevation; 2 = partial; 1 = vague; 0 = none.
- EX050 — Lesson. What kept the analysis focused? Rubric: 3 = pre-stated predictions; 2 = partial; 1 = vague; 0 = none.
Synthesis
Independent reproduction and honest answers strengthen a defense.
Continue studying — chapter 08 of 08
← Chapter 07 (draft) · Course chapter index · Worked results ↓ · Next course via chapter index →