Chapter 36 — Research methods

Lesson map

Lesson 36 published. Map, example, EX106.

flowchart TD
  N0["Learning outcomes"]
  N1["36.1 Research questions"]
  N2["36.2 Designs"]
  N3["36.3 Validity threats"]
  N4["36.4 Ethics"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX106"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["36.1 Research questions"]
  N2["36.2 Designs"]
  N3["36.3 Validity threats"]
  N4["36.4 Ethics"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX106"]

[Learning outcomes]
    |
    v
[36.1 Research questions]
    |
    v
[36.2 Designs]
    |
    v
[36.3 Validity threats]
    |
    v
[36.4 Ethics]
    |
    v
[Worked example] --> [Exercise EX106]

Learning outcomes

You will (1) choose research designs for governance questions; (2) consider validity threats; (3) plan ethical research.

36.1 Research questions

Amara learns to turn interests into research questions. Her idea becomes: do stores with data champions have lower rates of invalid entries than stores without, after accounting for differences between stores? She explains that a good question is specific, answerable with available data and useful to practitioners. Dr. Lindqvist helps her narrow it from a broad question about whether governance works, which would be impossible to answer in a dissertation. She writes the question at the top of her proposal draft and checks every later section against it.

36.2 Designs

She compares designs. A survey would show opinions but not effects. A before-and-after comparison in champion stores would show change but not whether champions caused it. Comparing champion and non-champion stores before and after introduction, a difference-in-differences design, accounts for changes affecting all stores. Shadish, Cook and Campbell discuss such quasi-experimental designs. She chooses this design, using synthetic data that mirror the patterns she observed during placement. She explains the choice to Dr. Lindqvist, who agrees that the design suits her question and the time available for a dissertation.

36.3 Validity threats

Amara lists threats to validity. Stores that volunteered champions may already have been more careful, a selection threat. Other changes, such as new scanners, may coincide with champions, a history threat. Different staff may record invalid entries differently, a measurement threat. She plans responses: comparing pre-existing trends, recording other changes and using the same automated rules for all stores. She notes that naming threats openly strengthens research. She writes each threat and its response into a table in the method chapter, so examiners can see how the design addresses it.

36.4 Ethics

Even with synthetic data, Amara considers ethics because her design mirrors real practices. She ensures that no real store or staff data appear and that her synthetic data generation is documented. If she later used real data, she would need university ethics approval and Harbourline's permission, and would aggregate results by store so individual staff could not be identified. Dr. Lindqvist notes that thinking through ethics early avoids problems later. Amara records these considerations in an ethics section of her proposal, which her tutor approves.

Worked example

Amara narrows her question to champions and invalid entries, chooses difference-in-differences on synthetic data, names selection, history and measurement threats and plans ethical safeguards.

Synthetic case. The design is synthetic; Shadish, Cook and Campbell (2002) is real.

Errors and limits

Before-and-after comparisons alone cannot separate effects from general trends.

Exercises

• EX106 — Chosen design. Which design did Amara choose?
  Show marking criteria

  Rubric: 3 = difference-in-differences;; 2 = partly correct;; 1 = other;; 0 = none.
• EX107 — Selection threat. What is the selection threat here?
  Show marking criteria

  Rubric: 3 = volunteer stores may already be more careful;; 2 = partly correct;; 1 = vague;; 0 = none.
• EX108 — Real data needs. What would real data require?
  Show marking criteria

  Rubric: 3 = ethics approval and Harbourline permission;; 2 = both;; 1 = one;; 0 = none.