Chapter 25 — Laboratory I
Lesson map
Lesson 25 published. Map, example, EX073.
flowchart TD
N0["Learning outcomes"]
N1["25.1 Planning the test"]
N2["25.2 Measuring"]
N3["Lab — Task time for forwarding versus button"]
N4["25.3 Reporting results"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX073"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["25.1 Planning the test"]
N2["25.2 Measuring"]
N3["Lab — Task time for forwarding versus button"]
N4["25.3 Reporting results"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX073"]
[Learning outcomes]
|
v
[25.1 Planning the test]
|
v
[25.2 Measuring]
|
v
[Lab — Task time for forwarding versus button]
|
v
[25.3 Reporting results]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX073]
Learning outcomes
You will (1) run a contained test of the report button with volunteers; (2) measure task time and success; (3) report results with limits.
25.1 Planning the test
In the first laboratory, Nadia tests the report button with ten staff volunteers using a test mailbox filled with synthetic emails. No real inboxes are used and volunteers know it is a test of the button, not of them. Each volunteer is asked to report two suspicious emails, once by forwarding to the address and once with the button. She records task time and whether the report succeeded. The plan has ethics approval and volunteers can stop at any time. Each session takes ten minutes.
25.2 Measuring
The lab compares task times. Forwarding takes about forty seconds on average, because people must find the address and add a note. The button takes about six seconds. All ten volunteers succeed with the button; two fail to forward correctly because they mistype the address. Nadia also asks volunteers to rate ease on a five-point scale. Ratings favour the button strongly. The test shows that a small design change removes much of the effort that discouraged reporting. She records every time in a table.
Lab — Task time for forwarding versus button
# Task times in seconds for ten volunteers (synthetic).
fwd = [35, 42, 38, 51, 33, 44, 39, 36, 47, 35]
btn = [5, 7, 6, 6, 5, 8, 6, 5, 7, 5]
print(f"forwarding mean {sum(fwd) / 10:.0f} s; button mean {sum(btn) / 10:.0f} s; saving {sum(fwd) / 10 - sum(btn) / 10:.0f} s")
Run with python3 lab_button.py. Verified output:
forwarding mean 40 s; button mean 6 s; saving 34 s
25.3 Reporting results
Nadia reports results in a short note with a table of times and success rates. She adds limits: volunteers may be more confident than typical staff; the test used a desktop computer and not phones; and ten people is a small sample. She recommends rolling out the button with a check of real reporting times afterwards. Stating limits honestly makes managers more likely to trust the result, because they can see what it does and does not show. Managers accept the note.
Worked example
Nadia tests a report button with ten volunteers in a test mailbox, finding a large time saving, and reports limits.
Synthetic case. The test is synthetic.
Errors and limits
Volunteers are not typical. Lab conditions differ from busy counters.
Exercises
- EX073 — Test mailbox. Why use a test mailbox? Rubric: 3 = so no real inboxes are involved;; 2 = contained;; 1 = vague;; 0 = none.
- EX074 — Mean times. What were mean times for forwarding and the button? Rubric: 3 = 40 and 6 seconds;; 2 = 34 seconds saved;; 1 = vague;; 0 = none.
- EX075 — Forwarding failures. How many volunteers failed to forward correctly? Rubric: 3 = two;; 2 = 2;; 1 = vague;; 0 = none.
Synthesis
Contained usability tests show the effect of small design changes and their limits.
Continue studying — chapter 25 of 40
← Chapter 24 (draft) · Course chapter index · Chapter 26 (draft) →