Chapter 5 — Checking data with Python
Lesson map
Lesson 05 published. Map, example, EX017.
flowchart TD
N0["Learning outcomes"]
N1["5.1 Reading a file"]
N2["5.2 Simple checks"]
N3["5.3 Summaries"]
N4["5.4 Saving and sharing"]
N0 --> N1 --> N2 --> N3 --> N4
N4 --> EX["Worked example"]
EX --> Q["Exercise EX017"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["5.1 Reading a file"]
N2["5.2 Simple checks"]
N3["5.3 Summaries"]
N4["5.4 Saving and sharing"]
N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX017"]
[Learning outcomes]
|
v
[5.1 Reading a file]
|
v
[5.2 Simple checks]
|
v
[5.3 Summaries]
|
v
[5.4 Saving and sharing]
|
v
[Worked example] --> [Exercise EX017]
Learning outcomes
You will (1) read a CSV file in Python; (2) write simple checks; (3) print a clear summary.
5.1 Reading a file
Mr. Osei introduces Python for repetitive checks. Marta writes a short script using the csv module to read a synthetic export of passenger counts. Each row has the date, route, trip time, boardings and alightings. She prints the number of rows and the first few rows to confirm the file was read correctly. She learns that checking the file reads properly is the first step before trusting any results from it. She also checks that the number of rows printed matches the number shown by the system export screen.
5.2 Simple checks
Marta writes checks: boardings and alightings must be whole numbers, not negative and not above one hundred and twenty, the bus capacity. The route must be one of Linkway's twelve routes. She counts rows failing each check. She notes that each check is a single clear rule, which makes it easy to explain to Mr. Hughes and to change if Linkway buys larger buses. She puts the capacity value at the top of the script as a named setting, so it can be changed in one place if Linkway ever buys larger buses.
5.3 Summaries
Marta prints a summary: rows read, rows failing each check and the share failing. She finds a few rows with negative alightings, which Mr. Hughes says come from a sensor fault on one bus. She adds the bus number to the output so the fault can be traced. She learns that a good summary helps someone act, not just know that something is wrong. Mr. Hughes passes the bus number to the maintenance team, who replace the faulty sensor that week, and the negative values stop appearing in later files.
5.4 Saving and sharing
Marta saves the script with comments explaining each step and the date. Mr. Osei shows her how to keep scripts in a shared folder with a short readme. When a classmate runs the script on a new month's file, it works without changes. Marta notes in her workbook that a script she can rerun saves hours compared with checking a spreadsheet by eye each month. Mr. Hughes asks for a copy of the script for Linkway's data team, and Marta adds a short note on how to run it.
Worked example
Marta reads a counts file with csv, checks boardings against capacity and valid routes, adds bus numbers to trace a sensor fault and saves a commented script others can rerun.
Synthetic case. Exports are synthetic.
Errors and limits
Trusting results before confirming the file read correctly risks wrong conclusions.
Exercises
- EX017 — Capacity limit. What upper limit did the check use? Rubric: 3 = one hundred and twenty;; 2 = partly correct;; 1 = wrong;; 0 = none.
- EX018 — Negative cause. What caused negative alightings? Rubric: 3 = a sensor fault on one bus;; 2 = partly correct;; 1 = vague;; 0 = none.
- EX019 — Traceability addition. What did Marta add to trace faults? Rubric: 3 = the bus number;; 2 = partly correct;; 1 = vague;; 0 = none.
- EX020 — Script sharing. How are scripts shared? Rubric: 3 = a shared folder with a readme;; 2 = partly correct;; 1 = vague;; 0 = none.
Continue studying — lesson 05 (chapter 05 of 18)
← Chapter 04 · Course chapter index · Chapter 06 →
Continue to the next authored lesson.