Chapter 3 — Original contribution proposal

Lesson map

Lesson 03 published. Map, example, EX015.

flowchart TD
  N0["Learning outcomes and connection"]
  N1["3.1 Types of contribution"]
  N2["3.2 The mechanism: complementary visibility"]
  N3["3.3 Observer model"]
  N4["3.4 Hypotheses and thresholds"]
  N5["3.5 Baselines and rivals"]
  N6["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX015"]

Logical drawing

flowchart LR
  N0["Learning outcomes and connection"]
  N1["3.1 Types of contribution"]
  N2["3.2 The mechanism: complementary visibility"]
  N3["3.3 Observer model"]
  N4["3.4 Hypotheses and thresholds"]
  N5["3.5 Baselines and rivals"]
  N6["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX015"]

[Learning outcomes and connection]
    |
    v
[3.1 Types of contribution]
    |
    v
[3.2 The mechanism: complementary visibility]
    |
    v
[3.3 Observer model]
    |
    v
[3.4 Hypotheses and thresholds]
    |
    v
[3.5 Baselines and rivals]
    |
    v
[Chapter synthesis]
    |
    v
[Worked example] --> [Exercise EX015]

Learning outcomes and connection

After this chapter you can (1) classify a proposed contribution by type; (2) express a mechanism that explains why a method should work; (3) write an observer model — the counterpart, in inventory research, of an attacker model; (4) state hypotheses with pre-registered thresholds and failure conditions; and (5) select fair baselines and name rival explanations. It turns the falsifiable gap of Chapter 2 into a proposal that Chapter 4 can pilot.

3.1 Types of contribution

A doctoral contribution in this area is usually one of: a new method (a dual-source inventory procedure), a new measurement (the first independent recall estimate for inventory tools), a new artefact (an open adjudicated corpus), a negative result (showing that a widely used metric is invalid), or a theory (a model of how agility reduces time-to-replace). State one primary type; secondary contributions are welcome but must not dilute the claim examiners will test. The Ciphertrail Study's primary contribution is a measurement — independent recall and precision of a dual-source method — with the adjudicated corpus as a secondary artefact.

3.2 The mechanism: complementary visibility

A method deserves study when there is a reason it should work. Static analysis — scanning source code, dependency manifests, binaries and configuration — sees cryptography the organization builds and configures, but not what closed vendor appliances do internally. Passive telemetry — handshake metadata from TLS, SSH and IKE — sees what is actually negotiated on the wire, including inside vendor appliances, but sees nothing about data at rest and nothing about devices that never talk on a monitored segment. The mechanism is complementary visibility: the two sources fail on different asset classes, so their union should raise recall where one is blind and the other is not, and should not help where both are blind. A mechanism that predicts where the method will fail is more valuable than one that predicts only success, because it makes the claim falsifiable.

3.3 Observer model

Define what each instrument can and cannot observe, as an attacker model would define capabilities:

Lesson table; scroll horizontally to read all columns →

Note the TLS 1.3 subtlety: certificates travel encrypted in TLS 1.3, so a purely passive sensor sees the selected key-exchange group but not the server's signature algorithm unless endpoint logging is added. A proposal that ignored this would overclaim passive coverage of signatures.

3.4 Hypotheses and thresholds

Pre-register, before seeing the main corpus:

• H1 — Dual-source recall of quantum-vulnerable uses exceeds static-only recall by at least 0.10 on the HRHN corpus (paired comparison; the lower bound of the 95% CI of the difference is at least 0.10).
• H2 — Dual-source precision is at least 0.90 (passive findings can produce duplicates and misattributions).
• H3 — The recall gain is concentrated in vendor appliances; in data-at-rest uses the gain is below 0.02. This hypothesis predicts a failure.
• H4 — Results transfer: on the untouched Marrow corpus, with thresholds and fusion rules frozen, the recall gain remains at least 0.05.

Each hypothesis has a stated condition under which it is rejected. H3 is included because a method that "helps everywhere" would be suspicious given the mechanism.

3.5 Baselines and rivals

The baseline is static-only analysis configured by a competent engineer with the same access, not a deliberately weak scanner. A second baseline is passive-only. Rival explanations for an observed gain include: passive findings duplicating static findings under different names (inflating counts), the adjudicators being influenced by which tool found an item (inflating precision), and the corpus over-representing appliances (inflating aggregate gain). The design answers each: canonical identifiers per use, blinded adjudication, and per-class reporting.

Write each rival with the result that would support it, so the design can be judged before data arrive.

3.6 Worked example and Lab 3

The proposal includes a model study to check whether H1 and H3 are even plausible before any data are collected. Detection probabilities are arbitrary teaching values chosen to embody the mechanism.

# Lab 3 - Ciphertrail Study (synthetic): static-only vs dual-source inventory recall.
# Each quantum-vulnerable "use" belongs to an asset class; each source detects it with a
# class-specific probability (ARBITRARY teaching values). Dual-source = union of detections.
import random
random.seed(2026)
classes = {  # class: (count, P(static detects), P(passive detects))
    "in-house code":       (420, 0.90, 0.55),
    "vendor appliance":    (260, 0.20, 0.80),
    "embedded/offline":    (180, 0.30, 0.10),
    "data at rest":        (140, 0.70, 0.00),
}
tot = {"static": 0, "dual": 0}; n_all = 0
print(f"{'class':18s}{'n':>5s}{'static':>8s}{'dual':>7s}")
for c, (n, ps, pp) in classes.items():
    s = d = 0
    for _ in range(n):
        hs, hp = random.random() < ps, random.random() < pp
        s += hs; d += (hs or hp)
    tot["static"] += s; tot["dual"] += d; n_all += n
    print(f"{c:18s}{n:5d}{s/n:8.2f}{d/n:7.2f}")
print(f"{'ALL':18s}{n_all:5d}{tot['static']/n_all:8.2f}{tot['dual']/n_all:7.2f}")

Expected output:

class                 n  static   dual
in-house code       420    0.89   0.94
vendor appliance    260    0.18   0.84
embedded/offline    180    0.39   0.47
data at rest        140    0.69   0.69
ALL                1000    0.59   0.79

The model shows the predicted pattern: a large gain on vendor appliances, almost none on data at rest, and an aggregate gain (0.20) that depends heavily on the class mix. That last point is a design warning — an organization with fewer appliances would show a smaller aggregate gain for the same method, so the thesis must report per-class results and the mix.

3.7 Common errors and limits

• Proposing a method without a mechanism, so that any result can be rationalized.
• Hypotheses that cannot fail, such as "dual-source recall is at least as high as static" (true by construction for a union).
• Weak baselines chosen to make the method look good.
• Ignoring that TLS 1.3 hides certificates from passive observers.
• Model studies prove only internal consistency; Lab 3's numbers are assumptions.

Each error has a visible symptom in a proposal: a method described without saying where it should fail, a hypothesis that a union guarantees, a static baseline run with default settings by someone unfamiliar with it, or a passive coverage claim for signatures under TLS 1.3. Check the proposal for those symptoms before the committee does.

Exercises — chapter 3

• EX015 — Contribution type. Classify the Ciphertrail contribution and one alternative framing; argue which an examiner would find stronger. Rubric: 3 = justified with reference to testability.
• EX016 — Mechanism. Write a mechanism paragraph for a different method (e.g. inventory from software bills of materials). Rubric: 3 = predicts at least one failure class.
• EX017 — Observer model. Add an observer "E — endpoint logs" to the table in 3.3. Rubric: 3 = sees/does-not-see columns accurate, including privacy cost.
• EX018 — Failing hypothesis. Rewrite "dual-source is better" as a hypothesis that could be rejected. Rubric: 3 = threshold, comparison and rejection condition stated.
• EX019 — Class mix. Modify Lab 3 so vendor appliances are 60 instead of 260 (adjust in-house code to 620). Report aggregate gain and explain. Rubric: 3 = correct run and interpretation tied to class mix.
• EX020 — Rivals. Name two further rival explanations for a recall gain and the design feature that addresses each. Rubric: 3 = rivals plausible, controls specific.

Chapter synthesis

A contribution is defensible when its mechanism predicts both success and failure, its observer model states what each instrument cannot see, its hypotheses can be rejected, and its baselines are strong. The Ciphertrail Study proposes complementary visibility and predicts where it will not help.

Continue studying — chapter 03 of 08

← Chapter 02 (draft) · Course chapter index · Chapter 04 (draft) →