Chapter 16 — Reference and master data
Lesson map
Lesson 16 published. Map, example, EX046.
flowchart TD
N0["Learning outcomes"]
N1["16.1 Two kinds of shared data"]
N2["16.2 Code lists"]
N3["16.3 Master data duplicates"]
N4["16.4 Golden records"]
N0 --> N1 --> N2 --> N3 --> N4
N4 --> EX["Worked example"]
EX --> Q["Exercise EX046"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["16.1 Two kinds of shared data"]
N2["16.2 Code lists"]
N3["16.3 Master data duplicates"]
N4["16.4 Golden records"]
N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX046"]
[Learning outcomes]
|
v
[16.1 Two kinds of shared data]
|
v
[16.2 Code lists]
|
v
[16.3 Master data duplicates]
|
v
[16.4 Golden records]
|
v
[Worked example] --> [Exercise EX046]
Learning outcomes
You will (1) distinguish reference and master data; (2) manage code lists; (3) handle duplicate master records.
16.1 Two kinds of shared data
Reference data are code lists used to classify other data, such as store codes, product categories and country codes. Master data describe core entities shared across systems, such as members, products and suppliers. Amara notes that both are used by many processes, so errors spread widely. DAMA-DMBOK2 treats them as a distinct management area. At Harbourline, each system keeps its own copies, and nobody checks whether those copies agree with each other over time. Amara lists each copy and where it lives.
16.2 Code lists
Amara finds that the store list exists in the till system, the loyalty system and finance, with differences: a closed store still active in loyalty, a new store missing from finance and one store with two codes. She proposes one authoritative list owned by operations, with changes approved and distributed to all systems. Each code has a start and end date so historical data still make sense after stores open, close or move. Operations accepts ownership and appoints a steward for the list.
16.3 Master data duplicates
Members sometimes register twice, using different cards or email addresses. Duplicates split purchase histories and send offers twice. Amara studies matching methods, from exact comparisons to probabilistic approaches based on Fellegi and Sunter's record linkage theory, which weighs agreement on several fields. She proposes a simple matching rule using name, postcode and birth year, with possible matches reviewed by a steward before merging, since wrong merges combine different people's data. The steward reviews a queue of possible matches each week and records each decision.
16.4 Golden records
After matching, Harbourline needs a single trusted version of each member, sometimes called a golden record. Amara proposes rules for choosing values when duplicates disagree: the most recently confirmed email, the address from the latest purchase delivery and the earliest joining date. Each rule is documented and approved by the loyalty owner. She notes that survivorship rules are governance decisions, because they determine which facts the organisation will treat as true. The rules are tested on synthetic duplicates before being applied to the live loyalty system.
Worked example
Amara separates reference and master data, finds three disagreeing store lists and proposes one owned list with dates, designs member matching reviewed by a steward and documents survivorship rules.
Synthetic case. Data are synthetic; DAMA-DMBOK2 and Fellegi and Sunter (1969) are real.
Errors and limits
Automatic merges without review can combine different people.
Exercises
- EX046 — Store list issues. Name one store list difference. Rubric: 3 = closed store still active, new store missing or duplicate code;; 2 = one;; 1 = vague;; 0 = none.
- EX047 — Merge safeguard. What safeguard applies before merging? Rubric: 3 = steward review of possible matches;; 2 = partly correct;; 1 = vague;; 0 = none.
- EX048 — Survivorship email. Which email survives? Rubric: 3 = the most recently confirmed;; 2 = partly correct;; 1 = other;; 0 = none.
Continue studying — lesson 16 (chapter 16 of 40)
← Chapter 15 · Course chapter index · Chapter 17 →
Continue to the next authored lesson.