◇ OpenCyber Academy
Professional · Chapter 3 · English

Chapter 3 — Authorized practice and evidence handling

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 3 — Authorized practice and evidence handling

Lesson map

Lesson 03 published. Map, example, EX011.

flowchart TD
  N0["Learning outcomes"]
  N1["3.1 Authorisation"]
  N2["3.2 Safe environments"]
  N3["3.3 Evidence"]
  N4["3.4 Sensitive findings"]
  N5["3.5 Repeat and quantify"]
  N6["3.7b Storing evidence safely"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX011"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["3.1 Authorisation"]
  N2["3.2 Safe environments"]
  N3["3.3 Evidence"]
  N4["3.4 Sensitive findings"]
  N5["3.5 Repeat and quantify"]
  N6["3.7b Storing evidence safely"]
  N7["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX011"]

[Learning outcomes]
    |
    v
[3.1 Authorisation]
    |
    v
[3.2 Safe environments]
    |
    v
[3.3 Evidence]
    |
    v
[3.4 Sensitive findings]
    |
    v
[3.5 Repeat and quantify]
    |
    v
[3.7b Storing evidence safely]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX011]

Learning outcomes

You will (1) work only within written authorisation; (2) use synthetic data and isolated environments; (3) collect and preserve evidence properly; (4) handle sensitive findings responsibly; and (5) report through the right channels.

3.1 Authorisation

Before any test, have written permission from the system owner stating systems, activities, dates, data and contacts. Testing without it can break the law and harm customers, even when intentions are good. If you find something outside scope, stop and ask.

Sofia's authorisation named ClaimChat staging, two weeks and synthetic customers only. Keep the signed document with the test records, and check that the dates still cover the work before each session. If the product owner wants more testing, such as against production, ask for a new written approval rather than extending the old one by email or conversation.

3.2 Safe environments

Test on isolated copies with synthetic data. If production testing is ever approved, use dedicated test accounts and actions that cannot affect real customers.

At Meridian, ClaimChat staging is the isolated copy for testing, with synthetic customers created for each test plan. Check before each session that the environment is still isolated, because data can be copied in by mistake, as Sofia found when a real customer's name appeared in a staging log. Keep test accounts clearly named as synthetic, and never point the staging callback tool at a real phone line.

3.3 Evidence

Record date and time, system and version, steps, inputs, outputs and tool logs. Measure effects (a callback was booked) rather than impressions (the reply sounded odd). Keep originals unchanged; work on copies; note who handled evidence.

In Sofia's tests, the evidence of an injected urgent callback is the booking in the tool log, not the wording of ClaimChat's reply. Store evidence with the test ID, so any result can be traced back to its input and version. Also note the model version and prompt in use at the time.

3.4 Sensitive findings

If you see real personal data, a live vulnerability or signs of an actual attack, stop testing and escalate through the incident process. Do not share details beyond those who need them. Coordinate disclosure with vendors when their products are affected.

Sofia's discovery of a real customer's name in a staging log is an example: she stopped, reported it to her supervisor and the privacy officer, and resumed only after the data was removed. Record what you saw and when, without copying the data itself.

3.5 Repeat and quantify

Repeat tests to estimate rates, and report the uncertainty.

"""Lab S: reporting attack-success rates honestly: Wilson intervals,
seed variation and a paired bootstrap. Synthetic trial outcomes only."""
import math, random
def wilson(k, n, z=1.96):
    p = k/n; d = 1 + z*z/n
    c = (p + z*z/(2*n))/d; h = z*math.sqrt(p*(1-p)/n + z*z/(4*n*n))/d
    return c - h, c + h
for k, n in ((3, 20), (30, 200), (300, 2000)):
    lo, hi = wilson(k, n); print(f"successes={k}/{n} rate={k/n:.3f} wilson95=[{lo:.3f}, {hi:.3f}]")
# five seeds of a synthetic defence evaluation: per-seed attack success rate
rates = []
for seed in range(5):
    r = random.Random(seed); rates.append(sum(r.random() < 0.12 for _ in range(200))/200)
m = sum(rates)/5; sd = math.sqrt(sum((x-m)**2 for x in rates)/4)
print("per-seed rates:", " ".join(f"{x:.3f}" for x in rates), f"mean={m:.3f} sd={sd:.3f}")
# paired bootstrap: baseline vs defence on the same 300 synthetic prompts
r = random.Random(42)
base = [r.random() < 0.30 for _ in range(300)]
defn = [b and r.random() < 0.6 for b in base]   # defence blocks ~40% of successes
diffs = []
for _ in range(2000):
    idx = [r.randrange(300) for _ in range(300)]
    diffs.append(sum(base[i] for i in idx)/300 - sum(defn[i] for i in idx)/300)
diffs.sort()
print(f"baseline_asr={sum(base)/300:.3f} defended_asr={sum(defn)/300:.3f} "
      f"reduction_ci95=[{diffs[50]:.3f}, {diffs[1949]:.3f}]")

Expected output:

successes=3/20 rate=0.150 wilson95=[0.052, 0.360]
successes=30/200 rate=0.150 wilson95=[0.107, 0.206]
successes=300/2000 rate=0.150 wilson95=[0.135, 0.166]
per-seed rates: 0.110 0.130 0.135 0.105 0.145 mean=0.125 sd=0.017
baseline_asr=0.350 defended_asr=0.190 reduction_ci95=[0.120, 0.203]

Sofia's 11 of 40 result is reported with an interval of about 16–43%, which tells the team the problem is real but its exact rate is uncertain. Run enough cases to support the decision at hand, and report the number of trials beside every rate. Lab S shows how the interval narrows as trials increase. A single run proves little, because model outputs vary between attempts; repeat each case and count. Sofia counted urgent bookings across her 40 uploads rather than judging from one.

3.6 Worked example (synthetic Meridian case)

Sofia's authorisation covers ClaimChat staging for two weeks, synthetic customers only. She uploads 40 synthetic repair quotes containing hidden text asking for an urgent callback. 11 of 40 cause an urgent booking (interval about 16–43%). During testing she notices a real customer's name in a staging log — production data copied by mistake. She stops, reports it to her supervisor and the privacy officer, and continues only after the data is removed.

The stop shows that authorisation covers what to do when things go wrong, not only what to test.

3.7 Errors and limits

Each error puts people or the work at risk. Verbal permission cannot be checked later and does not protect you if a test causes harm. Real customer data in tests creates the very exposure being studied. Continuing after finding a live problem, as Sofia did not, can spread personal data further and delay the response. Check your test plan and conduct for all three. Correct any of them before the next session.

3.7b Storing evidence safely

Screenshots of ClaimChat responses can contain customer names, claim numbers and photos. Meridian stores test evidence in a restricted folder, uses synthetic customers wherever possible, and deletes evidence on the date written in the test plan.

Sofia's evidence from the ClaimChat tests used synthetic customers, so screenshots carry no real data. Record the deletion date in the test plan and confirm deletion when it happens. Keep evidence in an access-controlled folder for the test team, not in chat channels or personal drives, and set a deletion date agreed with the privacy officer. Evidence that contains real data by mistake should be deleted at once and the incident reported.

Exercises

Synthesis

Authorised, safe testing with careful evidence and responsible escalation is the foundation of trustworthy AI-security practice.

Continue studying — chapter 03 of 12

← Chapter 02 (draft) · Course chapter index · Chapter 04 (draft) →