Chapter 39 — Capstone evaluation

Chapter 39 — Capstone evaluation

Lesson map

Lesson 39 published. Map, example, EX115.

flowchart TD
  N0["Learning outcomes"]
  N1["39.1 Results"]
  N2["39.2 Interpretation"]
  N3["39.3 Next steps"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX115"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["39.1 Results"]
  N2["39.2 Interpretation"]
  N3["39.3 Next steps"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX115"]

[Learning outcomes]
    |
    v
[39.1 Results]
    |
    v
[39.2 Interpretation]
    |
    v
[39.3 Next steps]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX115]

Learning outcomes

You will (1) evaluate a redesigned process against predefined outcomes; (2) interpret results cautiously; (3) recommend next steps.

39.1 Results

Sofia compares the redesigned quarter with the previous two. Completion by the deadline rose from eight of twelve managers to eleven of twelve. Median days to complete fell from fourteen to six. Rights removed per hundred reviewed fell slightly, from four to three. The manager survey shows most found the lists clearer. She presents each result with the counts behind it, avoiding percentages that would exaggerate small numbers. Eleven of twelve, for instance, is clearer to a small board than a ninety-two percent that implies a large sample.

39.2 Interpretation

Sofia interprets cautiously. Completion and speed improved clearly, consistent with the redesign helping. The small fall in removals matches her predefined interpretation risk: earlier reviews had already cleaned up much excess access. Other explanations exist; the quarter was quieter than usual and managers knew they were being studied. With twelve managers and one quarter, she cannot rule these out. She states the evidence supports the redesign without proving it caused the improvement. Her predefined interpretation risk from chapter 35 is what lets her explain the fall in removals without special pleading.

39.3 Next steps

Sofia recommends that Elmfield keep the redesign, continue measuring for three more quarters and extend the approach to the housing system review. She suggests adding a check that approved removals actually happen. Elmfield's director accepts the recommendations. Sofia learns that evaluation is valuable even when results cannot prove cause, because it tells the organisation whether to continue, adjust or stop and what to watch for next. The check that approved removals happen addresses a gap in the redesign itself, since removal requests could still be ignored.

Worked example

Sofia finds completion rose to eleven of twelve and median time fell to six days, interprets cautiously and recommends continued measurement.

Synthetic case. Results are synthetic.

Errors and limits

Small samples and observation effects limit conclusions. Surveys are subjective.

Exercises

• EX115 — Completion change. How did completion by the deadline change? Rubric: 3 = from eight of twelve to eleven of twelve;; 2 = improved;; 1 = vague;; 0 = none.
• EX116 — Median change. How did median days to complete change? Rubric: 3 = from fourteen to six;; 2 = faster;; 1 = vague;; 0 = none.
• EX117 — Cause caution. Why can Sofia not claim the redesign caused the improvement? Rubric: 3 = quieter quarter and observation effects;; 2 = alternatives;; 1 = vague;; 0 = none.

Synthesis

Careful evaluation guides decisions even when it cannot prove cause.

Continue studying — chapter 39 of 40

← Chapter 38 (draft) · Course chapter index · Chapter 40 (draft) →