Chapter 1 — Research question and literature method
Lesson map
Lesson 01 published. Map, example, EX001.
flowchart TD
N0["Learning outcomes"]
N1["1.1 The case"]
N2["1.2 Why the question matters"]
N3["1.3 Searching systematically"]
N4["Lab — Screening records"]
N5["1.4 Reading the screen"]
N6["1.5 Defining measures"]
N7["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
N7 --> EX["Worked example"]
EX --> Q["Exercise EX001"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["1.1 The case"]
N2["1.2 Why the question matters"]
N3["1.3 Searching systematically"]
N4["Lab — Screening records"]
N5["1.4 Reading the screen"]
N6["1.5 Defining measures"]
N7["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX001"]
[Learning outcomes]
|
v
[1.1 The case]
|
v
[1.2 Why the question matters]
|
v
[1.3 Searching systematically]
|
v
[Lab — Screening records]
|
v
[1.4 Reading the screen]
|
v
[1.5 Defining measures]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX001]
Learning outcomes
You will (1) frame a master's question about passkey adoption and account recovery; (2) search and screen literature systematically; (3) define measures before collecting data.
1.1 The case
Jonah Reyes is a master's student at the fictional Marlow Bay University, supervised by Dr Ingrid Solberg. His industry partner is the fictional Halyard Insurance, about nine hundred staff, whose service desk handles around forty password reset requests a week. Each reset takes staff time, interrupts work and is a known target for social engineering, because a caller who convinces the desk can take over an account. Halyard plans to enrol staff in passkeys, a phishing-resistant sign-in method based on the W3C Web Authentication standard. Jonah's dissertation asks a focused question: does passkey enrolment reduce password reset requests without lowering sign-in success?
1.2 Why the question matters
Passwords are forgotten, reused and phished; Florêncio and Herley (2007) showed in a large study that users hold many accounts and reuse a small number of passwords across them. Resets are the visible cost of that burden. NIST SP 800-63B-4 describes phishing-resistant authenticators and explains why they resist attacks that trick users into revealing secrets. If passkeys remove most reasons to reset a password, the service desk saves time and an attack path narrows. But if passkeys cause sign-in failures, staff may be locked out more often, which would simply move the burden elsewhere. Both outcomes must be measured.
1.3 Searching systematically
Jonah plans his literature search before running it. He lists databases, search terms such as passkey, WebAuthn, FIDO2, password reset and help desk, and a date range. He records every search string, the date and the number of results, so another student could repeat the search exactly. Inclusion criteria are written in advance: published from 2015, reporting empirical or survey evidence rather than opinion, and set in workplaces rather than consumer websites or student samples. Applying criteria written beforehand stops him unconsciously including papers that agree with what he hopes to find.
Lab — Screening records
"""Screen synthetic literature records against inclusion criteria and count each stage (stdlib only)."""
recs = [("R01", 2019, "empirical", "workplace"), ("R02", 2012, "empirical", "web"), ("R03", 2021, "opinion", "workplace"),
("R04", 2022, "empirical", "workplace"), ("R05", 2016, "empirical", "student"), ("R06", 2023, "empirical", "workplace"),
("R07", 2020, "survey", "workplace"), ("R08", 2018, "empirical", "web"), ("R09", 2024, "empirical", "workplace"),
("R10", 2017, "opinion", "web"), ("R11", 2023, "empirical", "student"), ("R12", 2021, "empirical", "workplace")]
s1 = [r for r in recs if r[1] >= 2015]; s2 = [r for r in s1 if r[2] in ("empirical", "survey")]; s3 = [r for r in s2 if r[3] == "workplace"]
print(f"found={len(recs)} year>=2015={len(s1)} evidence={len(s2)} workplace={len(s3)}")
print("included:", [r[0] for r in s3])
Run with python3 lab_screen.py. Verified output:
found=12 year>=2015=11 evidence=9 workplace=6
included: ['R01', 'R04', 'R06', 'R07', 'R09', 'R12']
1.4 Reading the screen
Of twelve synthetic records, one is excluded for its date, two for being opinion pieces and three for studying web users or students rather than workplaces, leaving six. Jonah records the reason for every exclusion in a table, which forms the basis of a flow diagram in his literature chapter. Six included studies is a small base, so he notes that his review will be narrative rather than statistical. He also keeps excluded consumer studies in a separate list, because they may still help explain mechanisms, even though they do not answer his workplace question directly.
1.5 Defining measures
Jonah defines two primary measures before collecting data. The first is password reset requests per hundred active staff per week, normalised because headcount changes. The second is sign-in success rate: successful sign-ins divided by all attempts, from the identity provider's synthetic-style aggregate logs. A secondary measure is the number of lockouts needing desk help. Each measure has a written definition, a data source and a known weakness: reset counts depend on how desk staff categorise tickets, so he checks a sample of tickets by hand to confirm categories are applied consistently.
Worked example
Jonah frames the question, writes inclusion criteria, screens twelve synthetic records to six workplace studies and defines reset rate per hundred staff and sign-in success as primary measures.
Synthetic case. Jonah Reyes, Marlow Bay University, Halyard Insurance and all records and figures are synthetic.
Errors and limits
Screening without criteria written in advance invites selection of studies that confirm the hoped-for answer.
Exercises
- EX001 — Reset risk. Why are password resets a security concern as well as a cost? Rubric: 3 = social engineering can take over accounts via the desk;; 2 = social engineering;; 1 = cost only;; 0 = none.
- EX002 — Passkey basis. Which standard are passkeys based on? Rubric: 3 = W3C Web Authentication;; 2 = WebAuthn;; 1 = SAML;; 0 = none.
- EX003 — Search record. What must Jonah record for each search? Rubric: 3 = string, date and result count;; 2 = two;; 1 = vague;; 0 = none.
- EX004 — Screen counts. How many records remain after screening in the lab? Rubric: 3 = 6 of 12, with stage counts 11, 9, 6;; 2 = 6;; 1 = 12;; 0 = none.
- EX005 — Exclusion reasons. Why were three records excluded at the last stage? Rubric: 3 = they studied web users or students, not workplaces;; 2 = setting;; 1 = date;; 0 = none.
- EX006 — Normalisation reason. Why normalise resets per hundred staff? Rubric: 3 = headcount changes over time;; 2 = headcount;; 1 = vague;; 0 = none.
- EX007 — Ticket check. How does Jonah check reset-count reliability? Rubric: 3 = hand-check a sample of ticket categories;; 2 = sample;; 1 = trust;; 0 = none.
Synthesis
A focused question, a repeatable search with fixed criteria and measures defined in advance give a master's study a solid start.