◇ OpenCyber Academy
Doctoral research · Chapter 3 · English

Chapter 3 — Original contribution proposal

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 3 — Original contribution proposal

Lesson map

Lesson 03 published. Map, example, EX015.

flowchart TD
  N0["Learning outcomes"]
  N1["3.1 Contribution statement"]
  N2["3.2 Hypotheses"]
  N3["3.3 Sampling"]
  N4["3.4 Data"]
  N5["3.5 Threats to validity"]
  N6["3.6 Usefulness"]
  N7["Feasibility"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX015"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["3.1 Contribution statement"]
  N2["3.2 Hypotheses"]
  N3["3.3 Sampling"]
  N4["3.4 Data"]
  N5["3.5 Threats to validity"]
  N6["3.6 Usefulness"]
  N7["Feasibility"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX015"]

[Learning outcomes]
    |
    v
[3.1 Contribution statement]
    |
    v
[3.2 Hypotheses]
    |
    v
[3.3 Sampling]
    |
    v
[3.4 Data]
    |
    v
[3.5 Threats to validity]
    |
    v
[3.6 Usefulness]
    |
    v
[Feasibility]
    |
    v
[Worked example] --> [Exercise EX015]

Learning outcomes

You will (1) write a contribution statement that is new, testable and useful; (2) state hypotheses and their falsifiers; (3) plan data, sampling and analysis; (4) anticipate threats to validity.

3.1 Contribution statement

"This thesis provides the first matched comparison of vulnerable-version exposure between projects that do and do not use automated update tools in a package ecosystem, and explains residual lag through maintainer interviews, yielding evidence-based guidance for SSDF practice RV.2 (assess, prioritise and remediate vulnerabilities)." The claim "first" rests on the literature matrix, which found update-tooling effects reported mainly in industry blogs. Yusuf keeps the matrix in the thesis appendix so examiners can check the claim against his search. The statement was later qualified after the workshop, when the within-project comparison showed a smaller effect than the matched comparison.

3.2 Hypotheses

H1: projects using update bots have shorter median exposure. H2: among bot users, long exposure is explained by breaking-change fear, test gaps or pinned transitive dependencies. Each hypothesis states what result would count against it: H1 fails if the bootstrap interval for the difference includes zero. H2 has its own counter-evidence: if interviews show that bot users with long exposure simply had no time, rather than test or dependency problems, H2's explanations are weakened. The proposal panel added a further check, comparing each interviewee's stated reason with their project's history of reverted upgrades.

3.3 Sampling

Projects depending directly on quillog before the fix, matched by size, age and activity, so that busy projects are not compared with abandoned ones. Abandoned projects (no commit for a year) are analysed separately. Matching on size, age and activity tries to make the groups comparable apart from bot use. Yusuf records the matching variables for every pair, so examiners can see how close the matches were. Abandoned projects would show long exposure for reasons unrelated to tooling, which is why they are kept out of the main comparison and described separately.

3.4 Data

Registry metadata (versions, dates), manifests from public repositories, advisory records and bot configuration files. No code is executed and no system is probed. The data are public, but Yusuf still records the date each item was collected, because manifests and advisories change. Bot configuration files tell him whether a project used an update bot during the exposure window, not only today. Each source is listed in the data management plan with how it was obtained and where it is stored.

3.5 Threats to validity

Selection: projects that adopt bots may already be careful. Measurement: a manifest change may not reach production. External: one ecosystem may not generalise. Each threat has a mitigation: matching, a sample check of release notes, and a replication in a second synthetic ecosystem. The matching addresses selection only partly, which the workshop reviewers later pointed out; the within-project comparison added after critique is the stronger response. The release-note check estimates how often a manifest change actually reached a published release. A second synthetic ecosystem tests whether the analysis code behaves the same way on different data.

3.6 Usefulness

The Larchwood Foundation wants guidance it can give maintainers; Yusuf commits to a plain-language summary. The summary will tell maintainers what the evidence supports, such as treating failing bot pull requests as security tasks, and what it does not, such as how users downstream behave. Larchwood's practitioners review a draft so the wording fits what maintainers actually read. Its claims will use the same numbers and limits as the thesis. The summary is planned for release alongside the thesis, not after it.

Feasibility

The plan is costed in months: three for data collection, two for interviews, four for analysis and writing.

Worked example

The proposal panel challenges H2: "How will you know fear of breaking changes is the reason, not an excuse?" Yusuf adds a check: for each interviewee, compare their stated reason with their project's history of reverted upgrades.

Synthetic case. The proposal, panel and data plan are synthetic.

Errors and limits

Matching reduces but cannot remove selection effects; say so.

Exercises

Synthesis

A contribution is new, falsifiable, matched to threats and useful to someone named.

Continue studying — chapter 03 of 08

← Chapter 02 (draft) · Course chapter index · Chapter 04 (draft) →