Chapter 34 — Data governance: integration
Lesson map
Lesson 34 published. Map, example, EX100.
flowchart TD
N0["Learning outcomes"]
N1["34.1 Bringing pieces together"]
N2["34.2 Change across interfaces"]
N3["34.3 End-to-end testing"]
N4["34.4 Integration ownership"]
N0 --> N1 --> N2 --> N3 --> N4
N4 --> EX["Worked example"]
EX --> Q["Exercise EX100"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["34.1 Bringing pieces together"]
N2["34.2 Change across interfaces"]
N3["34.3 End-to-end testing"]
N4["34.4 Integration ownership"]
N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX100"]
[Learning outcomes]
|
v
[34.1 Bringing pieces together]
|
v
[34.2 Change across interfaces]
|
v
[34.3 End-to-end testing]
|
v
[34.4 Integration ownership]
|
v
[Worked example] --> [Exercise EX100]
Learning outcomes
You will (1) integrate governance controls across teams and systems; (2) manage change across interfaces; (3) test end to end.
34.1 Bringing pieces together
By this point Kofi has owners, rules, lineage, catalogue entries, reconciliation and service reporting, built in different systems by different teams. Integration means ensuring they work together: rule failures appear in the catalogue badges, lineage links to owners, reconciliation breaks reach the right queues and service reports include causes from incident reviews. He builds an integration map showing every connection between governance components and the system that implements each one. The map shows which connections are automatic and which depend on a person remembering a manual step, and those manual steps become candidates for automation.
34.2 Change across interfaces
Kofi introduces a change notice process for interfaces in the leakage chain. Any team changing a field, schedule or format must notify affected owners two weeks in advance, with a test file. When the metering vendor plans a firmware update that changes timestamp precision, the notice allows billing to adjust its parser before the change goes live. Previously such changes were discovered only after failures, when readings had already been lost or misassigned. The vendor accepts the process and nominates a contact for notices.
34.3 End-to-end testing
Kofi designs an end-to-end test that pushes a synthetic day of data from simulated meters and sensors through every system to the leakage figure. The test checks that expected values appear at each stage, that controls fire on injected errors and that catalogue badges update. The first run reveals that reconciliation results reach the break queue but not the catalogue, so the badge shows stale figures. He fixes the connection and reruns the test successfully. The test now runs after every release automatically.
34.4 Integration ownership
Kofi notes that integration problems sit between teams, so nobody owns them by default. He proposes that the data governance lead owns the end-to-end test and integration map, with each team owning its components. Prof. Varga explains that integration ownership prevents the common failure where every component works in isolation but the whole fails, and that someone must be accountable for how pieces work together across organisational lines. The governance board confirms this assignment and adds the end-to-end test result to its monthly dashboard of governance health.
Worked example
Kofi maps connections between governance components, introduces change notices that catch a timestamp change, builds an end-to-end test that finds a stale badge and assigns integration ownership.
Synthetic case. Systems are synthetic.
Errors and limits
Component tests passing does not mean the system works.
Exercises
- EX100 — Notice period. How much notice do interface changes need? Rubric: 3 = two weeks with a test file;; 2 = partly correct;; 1 = partial;; 0 = none.
- EX101 — End-to-end finding. What did the first end-to-end run reveal? Rubric: 3 = reconciliation results did not reach the catalogue;; 2 = partly correct;; 1 = vague;; 0 = none.
- EX102 — Integration owner. Who owns the end-to-end test? Rubric: 3 = the data governance lead;; 2 = partly correct;; 1 = other;; 0 = none.
Continue studying — lesson 34 (chapter 34 of 50)
← Chapter 33 · Course chapter index · Chapter 35 →
Continue to the next authored lesson.