Chapter 10 — Engineering ethics and safety

Lesson map

Lesson 10 published. Map, example, EX019.

flowchart TD
  N0["Learning outcomes and connection"]
  N1["10.1 Duties of an engineer"]
  N2["10.2 Safety and security together"]
  N3["10.3 Authorisation and law"]
  N4["10.4 Deciding under uncertainty"]
  N5["10.6 Practical activity"]
  N6["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX019"]

Logical drawing

flowchart LR
  N0["Learning outcomes and connection"]
  N1["10.1 Duties of an engineer"]
  N2["10.2 Safety and security together"]
  N3["10.3 Authorisation and law"]
  N4["10.4 Deciding under uncertainty"]
  N5["10.6 Practical activity"]
  N6["Chapter synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX019"]

[Learning outcomes and connection]
    |
    v
[10.1 Duties of an engineer]
    |
    v
[10.2 Safety and security together]
    |
    v
[10.3 Authorisation and law]
    |
    v
[10.4 Deciding under uncertainty]
    |
    v
[10.6 Practical activity]
    |
    v
[Chapter synthesis]
    |
    v
[Worked example] --> [Exercise EX019]

Learning outcomes and connection

After this chapter you can (1) apply professional codes of ethics to infrastructure security decisions; (2) explain the relationship between safety and security and where they conflict; (3) recognise legal and authorisation boundaries on testing and research; and (4) make and document an ethical decision under uncertainty, including when to escalate. It builds on Chapter 5 and frames every later chapter, particularly testing (Chapters 30 and 40) and practice (Chapter 43).

10.1 Duties of an engineer

Professional codes converge on a hierarchy: hold paramount the safety, health and welfare of the public; act within competence; be honest and objective; avoid conflicts of interest; protect confidential information. The NSPE Code of Ethics for Engineers makes public safety the first fundamental canon; the ACM Code of Ethics and Professional Conduct asks computing professionals to avoid harm and to be honest about system capabilities and limits. In critical infrastructure the public is concrete: patients on dialysis, households without water, people waiting for an ambulance. The duty is not satisfied by following instructions if the result is foreseeable public harm.

10.2 Safety and security together

Safety protects people and the environment from harm caused by the system; security protects the system from deliberate misuse. In operational technology the two interact. A security control can create a safety hazard — an automatic account lockout on an operator console during an emergency, or a patch that restarts a controller during a process run. A safety mechanism can be a security target: the TRITON/TRISIS malware targeted safety instrumented system controllers, as described in CISA's HatMan malware analysis report. Good practice treats both in one hazard analysis (Chapter 33) and gives safety functions independence from the networks and systems they protect.

10.3 Authorisation and law

Testing a system you do not own or have written permission to test can be a crime, whatever your intentions, and in infrastructure it can endanger people. Authorisation must be written, specific (systems, times, methods), from someone entitled to give it, and include stop conditions. Research involving people or operational data needs ethics review and data agreements. Vulnerabilities you find incidentally should go to the asset owner privately or through a coordinated disclosure programme such as CISA's CVD programme. This book's labs are synthetic for exactly these reasons.

10.4 Deciding under uncertainty

A practical ethical decision procedure: (1) state the decision and who is affected; (2) identify duties and the facts that bear on them; (3) list options, including escalation and refusal; (4) assess consequences for the most vulnerable stakeholders; (5) choose, document the reasoning and the dissent, and set a review point. Engineers rarely face clean dilemmas; more often the problem is pressure to sign off on something not yet shown to work. "Not verified" is an honest and sufficient answer.

In the worked example that follows, the TMRP verification engineer uses exactly these five steps.

10.5 Worked example: the deadline sign-off (synthetic)

TMRP's commissioning deadline is Friday. Test T3 (setpoint bounds) has failed: the PS4 controller accepts a setpoint outside the safe range when sent as a NaN value. The vendor proposes commissioning with a note to fix in the next release. As the verification engineer, you apply the procedure: affected people include hospital patients if PS4 behaves unpredictably; your duty is to public safety and honest reporting; options are (a) sign, (b) refuse, (c) commission with a compensating control — remote setpoint writes disabled at the gateway, local-only operation for PS4 — and a dated fix. You recommend (c), record T3 as failed, not waived, and require re-test before remote writes are enabled. The decision is documented and signed by the operations manager, who owns the operational risk.

10.6 Practical activity

Write a one-page authorisation template for a test of a synthetic system: scope, methods excluded, time window, stop conditions, contacts, data handling and sign-off.

Fill in the template for a bench test of LRWA's telemetry gateway rather than for a live station. Name who may sign it, list the methods that are excluded, such as any traffic towards real RTUs, and give stop conditions that a tester can recognise without interpretation. Include a contact who can halt the test at any time and a rule for what happens to captured data afterwards. A template that would let someone test PS4 during a high-demand period has failed its purpose.

10.7 Common errors and limits

• Treating ethics as a compliance checklist rather than reasoning about harm.
• "Verbal permission" for testing.
• Security controls designed without considering safety in emergencies.
• Signing off what has not been verified because a schedule demands it.
• Public disclosure of infrastructure weaknesses without coordination.

The deadline sign-off in the worked example shows several of these at once: pressure to commission, a vendor offering a verbal promise to fix later, and a failed test that could easily have been marked waived. The engineer avoided each one by recording the failure and agreeing a compensating control in writing.

Exercises — chapter 10

EX019 — Safety–security conflict. Objective: resolve a control that endangers operations. Inputs: proposal to lock operator HMI accounts after five failed logins. Steps: analyse the hazard, propose an alternative that preserves security intent, and state how you would verify it. Expected evidence: half-page analysis. Rubric: 3 = hazard clear (lockout during emergency), alternative maintains accountability and resists guessing (e.g., rate-limiting with alerting, physical-presence override, shift accounts with logging), verification stated; 2 = alternative weak; 1 = hazard only; 0 = none.

EX020 — Incidental finding. Objective: act within authorisation. Inputs: while studying a public utility's procurement documents for coursework, you notice an internet-facing management page of a pump controller visible in a screenshot, with the vendor and model shown. Steps: state what you must not do, what you should do and to whom, and what your coursework may say. Expected evidence: 150 words. Rubric: 3 = no access or probing, private notification to the owner or via CVD, coursework anonymised; 2 = one step missing; 1 = suggests "checking whether it is real"; 0 = none.

Chapter synthesis

Public safety comes first, honestly reported. Safety and security must be analysed together because each can defeat the other. Test only with written, specific authorisation; report incidental findings privately. Decide under uncertainty with a documented procedure, and treat "not verified" as an answer you are allowed — and obliged — to give.