Chapter 21 — Reference codes in practice
Lesson map
Lesson 21 published. Map, example, EX061.
flowchart TD
N0["Learning outcomes"]
N1["21.1 The check"]
N2["Lab — Store codes in 2,000 synthetic sales rows chec"]
N3["21.2 Results"]
N4["21.3 Causes"]
N5["21.4 Fixes at source"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX061"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["21.1 The check"]
N2["Lab — Store codes in 2,000 synthetic sales rows chec"]
N3["21.2 Results"]
N4["21.3 Causes"]
N5["21.4 Fixes at source"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX061"]
[Learning outcomes]
|
v
[21.1 The check]
|
v
[Lab — Store codes in 2,000 synthetic sales rows chec]
|
v
[21.2 Results]
|
v
[21.3 Causes]
|
v
[21.4 Fixes at source]
|
v
[Worked example] --> [Exercise EX061]
Learning outcomes
You will (1) validate codes against a reference list; (2) classify invalid codes by cause; (3) design fixes at the source.
21.1 The check
After operations takes ownership of the store list, Amara checks sales data against it. Every sales row carries a store code, and valid codes are S01 to S40. She writes a script that compares each code with the authoritative list and counts invalid codes by value. She explains that grouping invalid codes by value is more useful than a single error count, because different invalid values usually have different causes and different people who can fix them. The script runs after each nightly load.
Lab — Store codes in 2,000 synthetic sales rows checked against a reference list
"""Check store codes in 2,000 synthetic sales rows against a reference list (stdlib only)."""
import random, collections
random.seed(3)
ref = {f"S{i:02d}" for i in range(1, 41)}
pool = sorted(ref) + ["S41", "s07", "S7", "S00"]
w = [1]*40 + [0.6, 0.5, 0.4, 0.3]
rows = random.choices(pool, weights=w, k=2000)
bad = collections.Counter(c for c in rows if c not in ref)
print(f"rows {len(rows)}; invalid codes {sum(bad.values())} ({100*sum(bad.values())/len(rows):.1f}%)")
print("by code:", dict(sorted(bad.items())))
Run with python3 lab_codes.py. Verified output:
rows 2000; invalid codes 89 (4.5%)
by code: {'S00': 11, 'S41': 32, 'S7': 16, 's07': 30}
21.2 Results
The lab checks two thousand sales rows. Eighty-nine rows carry invalid codes, 4.5 percent. The invalid values are S41 in 32 rows, s07 in 30 rows, S7 in 16 rows and S00 in 11 rows. Amara notes that the result is not one problem but four, and that the largest single group, S41, is a code that does not yet exist on the reference list, which suggests a process gap rather than careless typing at stores. Amara shares the breakdown with operations, IT and the store managers concerned on the same day.
21.3 Causes
Amara investigates each value. S41 belongs to a new store that opened before operations added it to the list. The lowercase s07 and the short S7 both come from a store system that accepts free text on manual entries. S00 is a default value used by a test till that was never removed from the live network. Each cause has a different owner: operations for new codes, IT for input validation and the store manager for the test till. Each owner confirms the cause.
21.4 Fixes at source
Amara recommends that new store codes be added to the list before opening, as a step in the store opening checklist; that the store system accept only codes from a drop-down list; and that the test till be removed. Mapping S7 and s07 to S07 in the warehouse fixes past data but would not stop new errors. Dr. Lindqvist notes that fixing causes at the source prevents a permanent cleaning job downstream. All three source fixes are completed within a month, and new invalid codes stop appearing.
Worked example
Amara finds 89 invalid codes in 2,000 rows (4.5 percent) split into S41, s07, S7 and S00, traces each to a different cause and owner and recommends source fixes.
Synthetic case. Codes are synthetic; outputs come from lab_codes.py.
Errors and limits
Mapping bad codes downstream without fixing sources creates endless cleaning.
Exercises
- EX061 — Invalid share. What share of rows had invalid codes? Rubric: 3 = 4.5% (89 of 2,000);; 2 = partly correct;; 1 = partial;; 0 = none.
- EX062 — S41 cause. Why did S41 appear? Rubric: 3 = a new store opened before being added to the list;; 2 = partly correct;; 1 = vague;; 0 = none.
- EX063 — S00 cause. What produced S00? Rubric: 3 = a test till left on the live network;; 2 = partly correct;; 1 = vague;; 0 = none.
Continue studying — lesson 21 (chapter 21 of 40)
← Chapter 20 · Course chapter index · Chapter 22 →
Continue to the next authored lesson.