Chapter 1 — Research question and literature method
Lesson map
Lesson 01 published. Map, example, EX001.
flowchart TD
N0["Learning outcomes"]
N1["1.1 From worry to question"]
N2["1.2 Search method"]
N3["1.3 Grading sources"]
N4["1.4 The search log"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX001"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["1.1 From worry to question"]
N2["1.2 Search method"]
N3["1.3 Grading sources"]
N4["1.4 The search log"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX001"]
[Learning outcomes]
|
v
[1.1 From worry to question]
|
v
[1.2 Search method]
|
v
[1.3 Grading sources]
|
v
[1.4 The search log]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX001]
Learning outcomes
You will (1) turn a practitioner worry into a researchable question; (2) run a documented literature search with inclusion criteria; (3) grade sources by type and reliability; (4) keep a search log that someone else can repeat.
1.1 From worry to question
Port security managers in CPSG believe that sharing indicators "helps". Lucía converts that belief into a question with a population, an intervention, a comparison and an outcome: among port operators, does receiving indicators through CPSG's TAXII 2.1 collection shorten the time between an indicator's first publication and a block rule in the member's controls, compared with similar operators outside the group? That structure makes clear what data she needs and what she does not. The outcome, hours from publication to block, is the part port managers had never measured.
1.2 Search method
A literature method states databases, search strings, date limits, languages, and inclusion and exclusion rules. Lucía searches digital libraries for "threat intelligence sharing" AND (effectiveness OR evaluation OR measurement), 2011–2026, English and Spanish. She includes empirical studies and standards; she excludes marketing pages without method.
Her first run of that string returned 412 records. The Spanish-language limb matters for a ports question, because several port authorities in her region publish security guidance only in Spanish; leaving it out would bias the review towards English-speaking operators. The date limit starts in 2011, before STIX and TAXII 2.1, so that early sharing studies are included.
1.3 Grading sources
Standards such as NIST SP 800-150 and the OASIS STIX/TAXII 2.1 specifications define practice and formats. Peer-reviewed studies test claims. Vendor reports describe observations under commercial incentives. Government guidance such as ICD 203 sets analytic norms. A literature chapter should keep these strands distinct so that a reader knows which claims are measured and which are recommended.
Lucía's 19 included works split along these lines: 4 standards, 9 peer-reviewed studies and 6 grey-literature reports. Only 3 of the studies measured an outcome rather than describing a process, so her literature chapter can say exactly how thin the measured evidence is, instead of letting six confident vendor reports stand in for it.
1.4 The search log
Each search gets a row: date, database, string, hits, screened, included, reason for exclusions. The log is an appendix, and it lets an examiner see that the review was not cherry-picked.
Lucía's log shows the funnel in numbers: 412 records, 300 removed on titles, 80 more on abstracts, and 19 included after full-text reading. Each removal step names its reason, such as "no measured outcome" or "marketing page without method". An examiner who doubts an exclusion can find the row and judge it, and another student can rerun the same strings on the same date range and compare the counts.
Worked example
Lucía's first string returns 412 records. Screening titles removes 300 off-topic items; abstracts remove 80 more; full-text reading leaves 19 included works: 4 standards, 9 peer-reviewed studies, 6 grey-literature reports. She notes that only 3 studies measure an outcome rather than describing a process — the start of her gap.
Synthetic case. CPSG and Lucía's search counts are invented.
Errors and limits
Searching only one database, or changing inclusion rules after reading results, biases a review. A master's review is systematic in method even when it is not a full systematic review.
Exercises
- EX001 — PICO shape. Label the population, intervention, comparison and outcome in Lucía's question. Rubric: 3 = all four correct; 2 = three; 1 = one or two; 0 = none.
- EX002 — Inclusion rule. Write one inclusion and one exclusion criterion for her review. Rubric: 3 = one clear criterion of each tied to question; 2 = one clear; 1 = vague; 0 = none.
- EX003 — Strand placement. Place NIST SP 800-150 in the right source strand and explain. Rubric: 3 = standard/guidance; defines practice, not measured outcomes; 2 = strand only; 1 = peer-reviewed study; 0 = none.
- EX004 — Log row. Write a complete search-log row for her first search. Rubric: 3 = date, database, string, hits, screened, included, exclusion reasons; 2 = five fields; 1 = two; 0 = none.
- EX005 — Funnel numbers. From 412 hits, how many remained after abstract screening? Rubric: 3 = 32; 2 = method right; 1 = wrong; 0 = none.
- EX006 — Gap seed. What observation from the screening suggests a research gap? Rubric: 3 = few studies measure outcomes (3 of 19); 2 = few studies; 1 = many studies; 0 = none.
- EX007 — Cherry-picking. Why must inclusion rules be fixed before screening? Rubric: 3 = prevents selecting sources to fit a conclusion; repeatability; 2 = fairness vaguely; 1 = speed; 0 = none.
Synthesis
Structure the question, document the search, keep source strands separate, and log every step so the review can be repeated.