Chapter 36 — Research methods

Lesson map

Lesson 36 published. Map, example, EX106.

flowchart TD
  N0["Learning outcomes"]
  N1["36.1 Research questions"]
  N2["36.2 Designs"]
  N3["36.3 Data and measures"]
  N4["36.4 Ethics"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX106"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["36.1 Research questions"]
  N2["36.2 Designs"]
  N3["36.3 Data and measures"]
  N4["36.4 Ethics"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX106"]

[Learning outcomes]
    |
    v
[36.1 Research questions]
    |
    v
[36.2 Designs]
    |
    v
[36.3 Data and measures]
    |
    v
[36.4 Ethics]
    |
    v
[Worked example] --> [Exercise EX106]

Learning outcomes

You will (1) frame a research question; (2) choose a suitable design; (3) plan ethical data collection.

36.1 Research questions

Aisha learns to frame research questions that can be answered with evidence. Her first idea, whether Ashfield's security is good, is too broad. She narrows it: did introducing an automated leaver process shorten the time leavers' accounts stay active? The question names an intervention, an outcome and a comparison. Dr. Carrow notes that narrow questions produce clear answers, while broad ones produce opinions. She writes the question at the top of every planning document, so each later decision about data and analysis can be checked against it. Dr. Carrow approves the wording.

36.2 Designs

Aisha considers designs. A randomised trial is impossible, because the leaver process applies to everyone. A before-and-after comparison is possible: account removal times for leavers in the year before the automated process and the year after. She learns from Shadish, Cook and Campbell's work on quasi-experiments that such comparisons are vulnerable to other changes happening at the same time, such as fewer leavers or a new IT staff member. She lists each rival explanation she can think of and plans a way to check it with available data.

36.3 Data and measures

Aisha defines her measure: days between a leaver's last working day, from human resources records, and account disabling, from system logs. She plans to use anonymised records, with Ashfield removing names before sharing. She will report median days and the share disabled within one day. She learns that defining measures precisely before collecting data prevents changing definitions later to get a preferred answer. She also decides how to treat leavers whose last working day is missing from the records: they will be excluded and counted separately, so readers know how many there were.

36.4 Ethics

The university ethics committee reviews her plan. Because records concern former staff, it requires anonymisation by Ashfield, storage on university systems only and reporting in aggregate. Aisha agrees. Dr. Carrow notes that even apparently harmless security data can relate to identifiable people, so ethics review protects both participants and researchers. The committee also asks her to explain how she will handle any evidence of misuse she discovers in the logs. She agrees to report it only to Ashfield's IT manager and not to include it in the dissertation, since investigating misuse is not her research purpose. The committee approves the plan with these conditions recorded.

Worked example

Aisha narrows her question to leaver removal time, chooses a before-and-after design while noting rival changes, defines the measure and gains ethics approval with anonymised data.

Synthetic case. The plan is synthetic.

Errors and limits

Changing measure definitions after seeing data biases results.

Exercises

• EX106 — Research question. What is Aisha's narrowed question?
  Show marking criteria

  Rubric: 3 = did the automated leaver process shorten removal time;; 2 = partly correct;; 1 = vague;; 0 = none.
• EX107 — Design weakness. What weakens a before-and-after design?
  Show marking criteria

  Rubric: 3 = other changes at the same time;; 2 = partly correct;; 1 = vague;; 0 = none.
• EX108 — Ethics condition. Name one ethics condition.
  Show marking criteria

  Rubric: 3 = anonymisation by Ashfield;; 2 = one;; 1 = vague;; 0 = none.