◇ OpenCyber Academy
Bachelor's · Chapter 37 · English

Chapter 37 — Evidence review seminar

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 37 — Evidence review seminar

Lesson map

Lesson 37 published. Map, example, EX073.

flowchart TD
  N0["Learning outcomes"]
  N1["37.1 Source types"]
  N2["37.2 Appraising timeline claims"]
  N3["37.3 Appraising performance claims"]
  N4["37.4 Structured critique"]
  N5["37.6 Practical activity"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX073"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["37.1 Source types"]
  N2["37.2 Appraising timeline claims"]
  N3["37.3 Appraising performance claims"]
  N4["37.4 Structured critique"]
  N5["37.6 Practical activity"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX073"]

[Learning outcomes]
    |
    v
[37.1 Source types]
    |
    v
[37.2 Appraising timeline claims]
    |
    v
[37.3 Appraising performance claims]
    |
    v
[37.4 Structured critique]
    |
    v
[37.6 Practical activity]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX073]

Learning outcomes

You will (1) critically appraise sources on quantum computing and PQC; (2) distinguish peer-reviewed results, standards, preprints, vendor material and journalism; (3) assess claims about timelines and performance; and (4) give and receive structured peer critique.

37.1 Source types

Standards (FIPS, SP, RFC) are authoritative for what algorithms are and how to use them, with status noted. Peer-reviewed papers carry expert scrutiny but can still be wrong. Preprints such as arXiv or IACR ePrint papers are early and unreviewed, though many are later published; Gidney (2025) was a preprint when cited here. Vendor material has commercial interests. Journalism simplifies and often misreports. Weight evidence accordingly and say which type you rely on.

A claim supported by two independent sources of different types is stronger than one repeated by many sources that copy each other.

37.2 Appraising timeline claims

Ask: who made the estimate, on what assumptions (error rates, qubit counts, algorithms), when, and what range they give? Distinguish resource estimates ("how many qubits would it take") from forecasts ("when will they exist"). Expert surveys provide distributions, not dates.

Check whether a newer estimate has replaced the one you are reading, since resource estimates have fallen sharply over time. When a news story gives a single year, trace it back to the original source and see what range and assumptions it actually stated. For HCU, timeline claims feed into Mosca's inequality, so a careless date can change which flows look urgent.

37.3 Appraising performance claims

Ask: what hardware, which implementation, which parameter set, what was measured (operations per second, handshake latency, bytes), and with what variability? Benchmarks of optimised assembly do not predict constrained-device behaviour.

Ask also whether the result has been reproduced by someone other than the authors. A performance figure measured on a server tells you little about a card terminal. For HCU, the useful benchmarks are those run on the same appliances and links used in the branches, so lab measurements often matter more than published tables.

37.4 Structured critique

Critique follows a structure: summarise the claim fairly; assess evidence; identify assumptions; note what would change the conclusion; suggest improvements. Critique the work, not the person.

A fair summary is the hardest step and the most important: if the author would not recognise your summary, the rest of the critique will miss. End with what the source does establish, not only its weaknesses. A critique that finds nothing useful in a source should say why it was considered at all.

37.5 Worked example

A vendor whitepaper states "our QKD system makes your network quantum-proof". Appraisal: vendor source; "quantum-proof" is not a defined property; QKD still needs authentication, which the paper does not address; NSA guidance does not support QKD for national security systems; the claim lacks scope. Conclusion: not adequate evidence for HCU's decision.

The appraisal still notes what the paper does show: the vendor has working hardware, which may matter for niche links. It simply does not answer HCU's question. A request for an independent evaluation would be the next step.

37.6 Practical activity

In groups, appraise one preprint, one standard and one news article on PQC using the frameworks above; present a two-minute critique each.

Use the same structure for each source: a fair summary, the evidence offered, the assumptions, and what would change the conclusion. Note the source type and how much weight it deserves. Then compare the three critiques as a group: did the standard, the preprint and the news article agree, and if not, which should a decision-maker trust and why?

37.7 Common errors and limits

Each lets weak evidence shape a decision it cannot support. Keep a short appraisal note for each source you rely on.

Exercises

Synthesis

Evidence review weighs sources by type and interest, interrogates assumptions behind timeline and performance claims, and critiques work constructively.

Continue studying — chapter 37 of 40

← Chapter 36 (draft) · Course chapter index · Chapter 38 (draft) →