Chapter 10 — Engineering ethics and safety
Lesson map
Lesson 10 published. Map, example, EX028.
flowchart TD
N0["Learning outcomes"]
N1["10.1 Duties"]
N2["10.3 Safe failure in an energy company"]
N3["10.4 Escalation"]
N4["10.5 Dual use"]
N5["Practice note — Professional codes"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
N6 --> EX["Worked example"]
EX --> Q["Exercise EX028"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["10.1 Duties"]
N2["10.3 Safe failure in an energy company"]
N3["10.4 Escalation"]
N4["10.5 Dual use"]
N5["Practice note — Professional codes"]
N6["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX028"]
[Learning outcomes]
|
v
[10.1 Duties]
|
v
[10.3 Safe failure in an energy company]
|
v
[10.4 Escalation]
|
v
[10.5 Dual use]
|
v
[Practice note — Professional codes]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX028]
Learning outcomes
You will (1) apply engineering-ethics duties to intelligence systems; (2) identify privacy and legal limits on OSINT; (3) design for safe failure in a critical-infrastructure context; (4) know when to escalate a concern.
10.1 Duties
Engineers owe duties to the public, the employer and the profession. In CTI that means protecting people whose data passes through the platform, honouring sharing agreements and refusing work that would facilitate harm.
For Kofi these duties are concrete. Personal data that arrives in a feed is limited to what analysis needs and deleted on schedule. Partner material marked AMBER stays within the agreed audience. If BEN asked for a capability that could hurt people, such as monitoring staff outside work, the duty to the public outweighs the instruction.
10.2 OSINT limits
Public does not mean unrestricted. Platform terms of service may forbid automated collection; data-protection law such as the EU GDPR applies to identifiable people even in public posts; aggregating harmless facts can create a profile that endangers someone. Lantern's OSINT module therefore collects organisational data — domains, certificates, published advisories — not personal social profiles.
Kofi checks each new OSINT source against its terms before writing any code, and records the outcome in the source register. A source that forbids automated access is read by hand, if at all, and nothing from it is stored in bulk.
10.3 Safe failure in an energy company
If Lantern pushes a bad block list to firewalls protecting substation telemetry, it could cut operational visibility. Safety design: operational-technology segments receive blocks only after human approval, and a malformed list causes the push to stop, not to clear the firewall.
Kofi's push service checks the list before sending it: it must parse, it must not be empty and it must not have shrunk by more than a set amount since the last push. A failed check stops the push and alerts the on-call engineer while the firewall keeps its current rules. For substation segments an operator approves each change, because a wrong block there can blind the control room.
10.4 Escalation
If a manager asks Kofi to scrape employee social accounts "to find insiders", he declines, documents the request and escalates to the privacy officer. Ethical refusal is part of engineering competence.
Writing down the request, who made it and what he did protects both Kofi and BEN. The privacy officer can then decide whether any legitimate concern lies behind it and how to address it lawfully, for example through BEN's existing insider-risk process. Kofi does not need to win the argument himself. He needs to make sure the right person hears it.
10.5 Dual use
Tooling that maps an adversary's infrastructure could also map a journalist's. Access controls, purpose limits and audit logs reduce misuse.
Kofi's design gives the infrastructure-mapping tools to a small group, requires a recorded purpose for each query and logs who searched for what. A monthly review samples the logs for searches that do not fit the purpose, such as names of people rather than domains. The controls do not make misuse impossible, but they make it visible, which deters most of it and allows the rest to be found.
Practice note — Professional codes
Engineering institutions publish codes of conduct that put public welfare first. In a security context those codes support refusal: a request that harms people or breaks the law is outside professional practice even if an employer insists. Kofi keeps a copy of his institution's code beside the privacy policy he references in design reviews.
Worked example
A request arrives to enrich every internal alert with the employee's home address "for context". Kofi asks what decision needs it; none does. The field is removed from the design and the decision recorded with the privacy officer's agreement.
Synthetic case. The request and decision are invented.
Errors and limits
Ethics is not a checkbox at the end of design. This chapter is not legal advice.
Exercises
- EX028 — Profile risk. Why can aggregating public facts about an employee be harmful even if each is public? Rubric: 3 = creates profile enabling stalking/harm; data-protection duties; 2 = privacy vaguely; 1 = harmless; 0 = none.
- EX029 — Fail-safe push. What should Lantern do if the generated block list for OT segments fails validation? Rubric: 3 = stop push, keep last good list, alert human; 2 = stop push; 1 = push empty list; 0 = none.
- EX030 — Insider scrape. Describe Kofi's correct response to the scraping request. Rubric: 3 = decline, document, escalate to privacy/legal; 2 = decline only; 1 = comply; 0 = none.
Synthesis
Protect people, honour terms and law, fail safely in critical environments, and escalate requests that cross lines.
Continue studying — chapter 10 of 50
← Chapter 09 (draft) · Course chapter index · Chapter 11 (draft) →