◇ OpenCyber Academy
Bachelor's · Chapter 18 · English

Chapter 18 — Research methods I

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 18 — Research methods I

Lesson map

Lesson 18 published. Map, example, EX052.

flowchart TD
  N0["Learning outcomes"]
  N1["18.1 Questions"]
  N2["18.2 Literature"]
  N3["18.3 Experiments"]
  N4["18.4 Honest reporting"]
  N5["18.4b Ethics in research design"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX052"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["18.1 Questions"]
  N2["18.2 Literature"]
  N3["18.3 Experiments"]
  N4["18.4 Honest reporting"]
  N5["18.4b Ethics in research design"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX052"]

[Learning outcomes]
    |
    v
[18.1 Questions]
    |
    v
[18.2 Literature]
    |
    v
[18.3 Experiments]
    |
    v
[18.4 Honest reporting]
    |
    v
[18.4b Ethics in research design]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX052]

Learning outcomes

You will (1) formulate research questions in AI security; (2) find and evaluate literature; (3) design simple experiments with controls; and (4) report results honestly with limitations.

18.1 Questions

A good question is specific, answerable and useful: "Does requiring patient confirmation reduce unauthorised bookings by Ask Lumen under injected documents, and what does it cost in completed bookings?" It names the system, the intervention, the outcome and the comparison.

The comparison is the part most often missing: without it, a result such as "no unauthorised bookings" cannot be attributed to confirmation rather than to the test cases being easy. Naming the cost, completed bookings, makes the study useful to LCH, which must weigh security against service.

18.2 Literature

Start from overviews — NIST AI 100-2 E2025 for adversarial ML terminology, OWASP LLM 2025 for LLM application risks, MITRE ATLAS for real-world techniques — then read key papers and note threat model, method, metrics and limitations. Verify every reference.

For her confirmation study, Rina reads work on indirect prompt injection and on human confirmation in automated systems, noting for each paper what attack it assumes and how success was measured. She records where papers disagree and why, often because of different threat models. Verifying references is the habit from the writing chapter, now applied to a full literature review.

18.3 Experiments

Change one thing at a time; keep a control condition; use held-out test data; repeat with several seeds; record versions. Decide metrics and thresholds before running, so results cannot steer the analysis.

In Rina's study, the one change is confirmation on or off; the control is the same chatbot without it; test documents are written separately from development; and the metrics, bookings caused by injected documents and normal bookings completed, are fixed in advance with a plan for intervals, written before any run.

18.4 Honest reporting

Report all conditions, not only the best; give intervals; describe what was not tested. Lipton and Steinhardt (arXiv:1807.03341) describe "troubling trends" such as unclear sources of gains; avoid them by explaining which change caused which effect.

Rina's report includes the drop in completed bookings, which some readers might see as weakening the case for confirmation. Leaving it out would mislead LCH about the cost. She also explains that confirmation, not any other change, caused the effect, because only that setting differed between conditions. What she did not test, such as other languages, is listed.

18.4b Ethics in research design

AI-security experiments can create harm: payloads that work on real products, datasets with personal data, or tests on systems you do not own. Use synthetic data and isolated systems, keep payloads harmless (for example aimed at reserved .invalid domains), obtain ethics approval when people participate, and plan disclosure if you discover a real vulnerability.

Rina's injected documents try to book appointments for a synthetic patient ID in the isolated copy, so nothing affects real patients or real systems. No clinic staff are deceived.

18.5 Worked example (synthetic LCH case)

Rina's first study compared Ask Lumen with and without confirmation on 300 synthetic injected documents and 300 normal booking conversations. Without confirmation, 21 injected documents caused bookings (7.0%); with confirmation, 0 (upper bound ≈ 1%). Normal bookings completed fell from 97% to 94% because some test users abandoned at the confirmation step. She reported both effects and noted that test users were classmates, not patients.

The 7.0% rate has a Wilson interval of about 4.6% to 10.5%, clearly above the confirmed condition, so the difference is not a chance result. The three-point drop in normal bookings is the cost.

18.6 Errors and limits

A question too broad, such as "is Ask Lumen secure?", cannot be answered by any single study. Without a control condition, an observed result cannot be attributed to the intervention. Reporting only favourable results, for example the security gain without the drop from 97% to 94% in completed bookings, misleads decision makers about trade-offs. Rina's study design avoids each of these by naming the comparison, the cost and the limits in advance.

Exercises

Synthesis

Research methods turn questions into trustworthy answers: clear questions, solid literature, controlled experiments and honest reports with limitations.

Continue studying — chapter 18 of 40

← Chapter 17 (draft) · Course chapter index · Chapter 19 (draft) →