Chapter 1 — Research question and literature method
Lesson map
Lesson 01 published. Map, example, EX001.
flowchart TD
N0["Learning outcomes and connection"]
N1["1.1 From interest to question"]
N2["1.2 Searching two communities"]
N3["1.3 Screening and agreement"]
N4["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4
N4 --> EX["Worked example"]
EX --> Q["Exercise EX001"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["1.1 From interest to question"]
N2["1.2 Searching two communities"]
N3["1.3 Screening and agreement"]
N4["Chapter synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX001"]
[Learning outcomes and connection]
|
v
[1.1 From interest to question]
|
v
[1.2 Searching two communities]
|
v
[1.3 Screening and agreement]
|
v
[Chapter synthesis]
|
v
[Worked example] --> [Exercise EX001]
Learning outcomes and connection
After this chapter you can (1) turn a vague interest into a researchable question about a control loop, (2) search across control engineering and security venues with synonyms, (3) deduplicate and screen with agreement metrics, and (4) write a scoping protocol. It prepares the seminar of Chapter 2.
1.1 From interest to question
"OT security" is a field, not a question. A researchable question names a system, a phenomenon, a comparison and a scope. Harbor Loop example: In a synthetic single-tank level loop, does a residual CUSUM using an independently wired flow meter detect single-sensor replay with higher event recall and lower false-alarm rate than a ±1.5 band alarm, and does the advantage vanish when the flow meter is also spoofed? The question already encodes the failure condition — a master's-level habit that saves months of overclaiming.
1.2 Searching two communities
Physics-based detection appears as "analytical redundancy" and "residual generation" in control journals and as "false data injection" and "sensor spoofing" in security venues. Build a synonym table before searching IEEE Xplore, ACM DL, Scopus and arXiv. Include grey literature (IEC white papers, CISA advisories) as a separate stream with its own eligibility rules, because it is not peer-reviewed in the same way.
Run each search string in every database and save the exports with the date, the exact query and the hit count, so the search can be repeated by an examiner. Expect the two communities to describe the same detector with different words, and expect duplicates across venues; Lab 1 shows how quickly they appear when one survey is indexed by both IEEE and ACM.
1.3 Screening and agreement
Two reviewers screen titles and abstracts independently; report Cohen's kappa. Lab 1 shows a synthetic export with DOI and title-year deduplication and a kappa of 0.462 — moderate agreement driven by disagreement on a vendor white paper and a DNP3 measurement paper. Low agreement is a signal to repair eligibility criteria, not to force consensus.
Write the eligibility rules as testable statements before screening: does the study evaluate a detector on a control loop, does it name its attacker model, does it report a metric beyond accuracy. Screen a small calibration batch together, discuss each disagreement, and only then screen the rest independently. Record each changed rule and the date it changed.
1.4 Worked example and Lab 1
# Lab 1 (masters OT, synthetic): deduplicate a search export and compute Cohen's kappa for two screeners.
import re
RECORDS=[
("ieee","Physics-based anomaly detection for ICS: a survey",2018,"10.1145/3203245"),
("acm","Physics-Based Anomaly Detection for ICS: A Survey",2018,"10.1145/3203245"),
("ieee","Modbus security: a survey of attacks and defenses",2020,""),
("arxiv","Sensor replay attacks on water treatment testbeds",2021,""),
("ieee","Sensor Replay Attacks on Water Treatment Testbeds",2021,""),
("grey","Vendor white paper: securing SCADA with AI",2022,""),
("ieee","DNP3 Secure Authentication field measurements",2019,""),
("acm","Zone and conduit design for IEC 62443",2020,""),
("ieee","False data injection in power systems: a survey",2011,"10.1109/TSG.2011.2129544"),
("arxiv","False Data Injection Attacks against State Estimation in Electric Power Grids",2011,"10.1109/TSG.2011.2129544"),
]
def key(r):
doi=r[3].strip().lower();
if doi: return ("doi",doi)
t=re.sub(r"[^a-z0-9]","",r[1].lower()); return ("title",t,r[2])
seen={}; kept=[]
for r in RECORDS:
k=key(r)
if k in seen: continue
seen[k]=True; kept.append(r)
# two reviewers: I=include X=exclude; disagreements on grey and DNP3
A="IIXIXIIXIX"[:len(kept)]; B="IIXIXXXIIX"[:len(kept)]
def kappa(a,b):
n=len(a); agree=sum(x==y for x,y in zip(a,b))/n
pa=sum(x=="I" for x in a)/n; pb=sum(x=="I" for x in b)/n
pe=pa*pb+(1-pa)*(1-pb)
return round(agree,3), round((agree-pe)/(1-pe) if pe!=1 else 1.0,3)
print("raw",len(RECORDS),"kept",len(kept),"dropped",len(RECORDS)-len(kept))
print("agree,kappa",kappa(A,B))
print("kept titles:",[r[1][:40] for r in kept])
Expected output:
raw 10 kept 7 dropped 3
agree,kappa (0.714, 0.462)
kept titles: ['Physics-based anomaly detection for ICS:', 'Modbus security: a survey of attacks and', 'Sensor replay attacks on water treatment', 'Vendor white paper: securing SCADA with ', 'DNP3 Secure Authentication field measure', 'Zone and conduit design for IEC 62443', 'False data injection in power systems: a']
Three duplicates dropped via DOI or normalized title. Kappa 0.462 with 71.4% raw agreement shows chance correction at work.
The lab keeps the first record of each duplicate pair, so the IEEE version of the physics-based survey stays and the ACM copy goes. Two records with no DOI are merged only because their normalized titles and years match exactly; a small change in wording would let a duplicate through. Check the kept list by eye before screening, and report raw count, kept count and dropped count in the protocol, as the lab prints them.
1.5 Common errors and limits
Searching only security venues; treating vendor papers as peer-reviewed evidence; reporting raw agreement without kappa; writing a question with no failure condition.
A question with no failure condition invites a dissertation that can only succeed. The Harbor Loop question avoids that by asking when the CUSUM advantage disappears. The search has a similar limit: databases change their indexing, so a search run in March may not return the same set in September. Record the date of every run and treat the corpus as a snapshot.
Exercises — chapter 1
- EX001 — Researchable Modbus question. Rewrite "I want to study Modbus security" as a researchable question with system, phenomenon, comparison and scope. Rubric: 3 = system, phenomenon, comparison, scope with ≥90% metric; 2 = missing one element; 1 = vague Modbus interest; 0 = none.
- EX002 — Synonym table for sensor manipulation. Build a synonym table (≥4 control + ≥4 security terms) for detecting manipulated sensor data. Rubric: 3 = ≥4 control + ≥4 security terms; 2 = 3+3; 1 = one community; 0 = none.
- EX003 — Include or exclude white paper and DNP3. Write inclusion/exclusion criteria that decide the vendor white paper and the DNP3 paper in Lab 1. Rubric: 3 = criteria exclude vendor marketing, include DNP3 field measurement; 2 = decisions without criteria; 1 = one decision; 0 = none.
- EX004 — Ampersand grey record and kappa. Run Lab 1; add a grey record with an ampersand in the title; show whether your key() catches it; recompute kappa after changing one decision. Rubric: 3 = normalize key catches ampersand + kappa movement explained; 2 = key without kappa; 1 = kappa only; 0 = none.
- EX005 — One-page scoping protocol. Draft a one-page scoping protocol (databases, dates, synonyms, screening rules, kappa target). Rubric: 3 = databases/dates/synonyms/screening/κ≥0.6; 2 = missing one field; 1 = bullet list; 0 = none.
- EX006 — DOI key before title key. Explain why a DOI key must precede a title key in deduplication. Rubric: 3 = DOI identity across title variants + title-key failure mode; 2 = DOI first without failure mode; 1 = slogan; 0 = none.
- EX007 — Evidence tiers for three sources. Classify three sources (Giraldo survey, CISA advisory, vendor blog) by evidence tier for a dissertation. Rubric: 3 = Giraldo high, CISA medium, vendor blog low with reasons; 2 = tiers without reasons; 1 = one source; 0 = none.
Chapter synthesis
A master's project starts with a question that already names its failure condition, a search that spans communities, and a screening process whose disagreements repair the criteria.