Chapter 14 — Privacy and data protection: automation
Lesson map
Lesson 14 published. Map, example, EX053.
flowchart TD
N0["Learning outcomes"]
N1["14.1 The retention rule"]
N2["14.2 The retention lab"]
N3["14.3 Exceptions and safety"]
N4["14.4 Monitoring the job"]
N5["Lab — Reminder records past retention"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX053"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["14.1 The retention rule"]
N2["14.2 The retention lab"]
N3["14.3 Exceptions and safety"]
N4["14.4 Monitoring the job"]
N5["Lab — Reminder records past retention"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX053"]
[Learning outcomes]
|
v
[14.1 The retention rule]
|
v
[14.2 The retention lab]
|
v
[14.3 Exceptions and safety]
|
v
[14.4 Monitoring the job]
|
v
[Lab — Reminder records past retention]
|
v
[Worked example] --> [Exercise EX053]
Learning outcomes
You will (1) automate a retention rule; (2) handle exceptions; (3) monitor the automation.
14.1 The retention rule
Appointment reminder records, which store a phone number, appointment time and delivery status, are needed for about a year to resolve complaints and check delivery problems. Marlow's retention schedule sets 365 days. Clinical records follow longer professional rules and are not part of this job. Leila writes the rule precisely: delete reminder records whose appointment date is more than 365 days before the run date, except those linked to an open complaint. A precise rule is easier to automate and to explain to auditors.
14.2 The retention lab
The lab applies the rule to 1,200 synthetic reminder records. It finds 712, 59.3%, older than 365 days, and the oldest record is 900 days old. The result shows that reminders have never been deleted since the SMS service started, so most stored reminders are already beyond their purpose. Leila points out that every excess record contains a phone number linked to a dental appointment, which would be exposed if the reminder database were breached, so deletion directly reduces risk. Deleting the 712 expired records removes most of the phone numbers the database holds without affecting any current appointment.
14.3 Exceptions and safety
Before enabling deletion, Leila adds an exception flag for records linked to open complaints, set by the complaints team. She runs the job first in report-only mode, listing what would be deleted without deleting anything, and the operations director reviews the counts. Only then is deletion switched on. The job writes a summary of counts deleted and held to a log each night, without listing phone numbers. She explains that report-only runs are a simple way to catch mistakes in rules before data are lost.
14.4 Monitoring the job
The job sends a daily success message with the counts. If no message arrives, the IT contractor receives an alert. Each quarter, Leila checks that the oldest reminder record is younger than 366 days and that held records have a current complaint reference. This monitoring addresses the silent failure mode from Chapter 8. Mr. Okafor notes that automation enforces storage limitation reliably, but only if somebody notices when the automation stops, which is why the success message matters as much as the job. Both checks take minutes.
Lab — Reminder records past retention
"""Apply a retention rule to synthetic appointment-reminder records (stdlib only)."""
import random, datetime
random.seed(11)
today = datetime.date(2026, 10, 1)
recs = [(i, today - datetime.timedelta(days=random.randint(0, 900))) for i in range(1200)]
keep_days = 365
expired = [r for r in recs if (today - r[1]).days > keep_days]
oldest = max((today - d).days for _, d in recs)
print(f"reminder records {len(recs)}; older than {keep_days} days {len(expired)} ({100*len(expired)/len(recs):.1f}%)")
print(f"oldest record {oldest} days")
Run with python3 lab_retention.py. Verified output:
reminder records 1200; older than 365 days 712 (59.3%)
oldest record 900 days
Worked example
Leila finds 712 of 1,200 reminder records (59.3%) older than 365 days, adds a complaint exception, runs report-only first, then enables deletion with daily success messages and quarterly checks.
Synthetic case. Records are synthetic.
Errors and limits
Backups may still hold deleted records until they expire; document backup retention.
Exercises
- EX053 — Past retention. How many reminder records were past retention? Rubric: 3 = 712 of 1,200, 59.3%;; 2 = approximate;; 1 = wrong;; 0 = none.
- EX054 — Oldest record. How old was the oldest record? Rubric: 3 = 900 days;; 2 = partly correct;; 1 = wrong;; 0 = none.
- EX055 — Report-only. Why run in report-only mode first? Rubric: 3 = catch rule mistakes before data are lost;; 2 = mentions mistakes;; 1 = vague;; 0 = none.
- EX056 — Exception. Which records are held? Rubric: 3 = those linked to open complaints;; 2 = partly correct;; 1 = vague;; 0 = none.
Continue studying — lesson 14 (chapter 14 of 18)
← Chapter 13 · Course chapter index · Chapter 15 →
Continue to the next authored lesson.