Chapter 27 — Analytics and governance

Chapter 27 — Analytics and governance

Lesson map

Lesson 27 published. Map, example, EX079.

flowchart TD
  N0["Learning outcomes"]
  N1["27.1 Governing outputs"]
  N2["27.2 Documenting models"]
  N3["27.3 Analytics risks"]
  N4["27.4 Self-service"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX079"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["27.1 Governing outputs"]
  N2["27.2 Documenting models"]
  N3["27.3 Analytics risks"]
  N4["27.4 Self-service"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX079"]

[Learning outcomes]
    |
    v
[27.1 Governing outputs]
    |
    v
[27.2 Documenting models]
    |
    v
[27.3 Analytics risks]
    |
    v
[27.4 Self-service]
    |
    v
[Worked example] --> [Exercise EX079]

Learning outcomes

You will (1) govern analytical outputs; (2) document models and reports; (3) manage risks from analytics.

27.1 Governing outputs

Analysts at Harbourline produce dashboards, forecasts and customer segments. Amara notes that outputs carry the quality and limitations of their inputs, plus new risks from analytical choices. She proposes that important outputs have owners, catalogue entries and documented definitions, just like source data. A sales forecast used to order stock, for example, needs a named owner who answers for its accuracy and explains changes in its method. Amara starts with the five outputs most used in decisions, including the stock forecast and the weekly sales dashboard.

27.2 Documenting models

Amara documents a model that predicts which members may stop shopping. The documentation covers purpose, data used, method, accuracy on test data, known weaknesses and approved uses. She notes that the model performs worse for members in rural stores, because they have fewer data. The documentation warns users not to rely on it there. She explains that documentation helps users apply models appropriately and helps reviewers find problems before they cause harm. The documentation is stored with the model's catalogue entry for all users.

27.3 Analytics risks

Risks include using personal data beyond agreed purposes, models that treat groups unfairly and dashboards that mislead through poor definitions. Amara proposes a short review for new analytical products, covering data sources, lawful basis, fairness checks and definitions. She tests the review on a new segment and finds it uses birth year, which is missing for many members, so the segment underrepresents older customers. The analyst adjusts the method. The review takes under an hour for most products, and analysts begin to request it themselves before launching new segments.

27.4 Self-service

Harbourline wants more staff to build their own reports. Amara supports this but warns that self-service without governance multiplies conflicting figures. She proposes certified data sets in the warehouse, with approved definitions, as the basis for self-service reports. Reports using certified data carry a mark. Dr. Lindqvist notes that self-service and governance work together when the governed path is also the easiest one for staff to follow. Amara suggests that certified data sets be the default view in the reporting tool, so staff use them without having to search for alternatives.

Worked example

Amara gives analytical outputs owners and catalogue entries, documents a churn model's rural weakness, reviews a segment that underrepresents older members and proposes certified data sets for self-service.

Synthetic case. Models are synthetic.

Errors and limits

Uncertified self-service reports multiply conflicting figures.

Exercises

• EX079 — Model weakness. Where does the churn model perform worse? Rubric: 3 = rural stores;; 2 = partly correct;; 1 = other;; 0 = none.
• EX080 — Segment problem. Why did the segment underrepresent older customers? Rubric: 3 = it used birth year, which is often missing;; 2 = partly correct;; 1 = vague;; 0 = none.
• EX081 — Self-service basis. What should self-service reports use? Rubric: 3 = certified data sets;; 2 = partly correct;; 1 = vague;; 0 = none.

Continue studying — lesson 27 (chapter 27 of 40)

← Chapter 26 · Course chapter index · Chapter 28 →

Continue to the next authored lesson.