◇ OpenCyber Academy
Technologist · Chapter 3 · English

Chapter 3 — Programming foundations

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 3 — Programming foundations

Lesson map

Lesson 03 published. Map, example, EX007.

flowchart TD
  N0["Learning outcomes"]
  N1["3.1 Defanging"]
  N2["Lab — refang and classify"]
  N3["3.2 Reading the result"]
  N4["3.3 Text only"]
  N5["3.4 Structure"]
  N6["3.5 Testing edges"]
  N7["3.6 Comments and names"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX007"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["3.1 Defanging"]
  N2["Lab — refang and classify"]
  N3["3.2 Reading the result"]
  N4["3.3 Text only"]
  N5["3.4 Structure"]
  N6["3.5 Testing edges"]
  N7["3.6 Comments and names"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX007"]

[Learning outcomes]
    |
    v
[3.1 Defanging]
    |
    v
[Lab — refang and classify]
    |
    v
[3.2 Reading the result]
    |
    v
[3.3 Text only]
    |
    v
[3.4 Structure]
    |
    v
[3.5 Testing edges]
    |
    v
[3.6 Comments and names]
    |
    v
[Worked example] --> [Exercise EX007]

Learning outcomes

You will (1) write small Python scripts that handle indicators as text; (2) refang defanged values safely; (3) validate addresses with the standard library; (4) keep scripts free of network activity.

3.1 Defanging

Analysts write "pay-portal[.]example" or "hxxps://" so that links in reports cannot be clicked by accident. Scripts must convert these back to normal form — refang — before matching against logs, but only as text.

The square brackets break the domain so that a mail client or document viewer does not turn it into a link. Refanging reverses that only inside the script, where the value is compared with log lines as plain text. In the lab, replace("[.]", ".") and replace("hxxp", "http") do the conversion, and nothing in the code opens a connection to the resulting host.

Lab — refang and classify

"""Lab (ch 03): refang defanged indicators safely as text and classify them (synthetic; no network)."""
import ipaddress
raw = ["hxxps://pay-portal[.]example/login", "198.51.100[.]23", "update-svc[.]example", "203.0.113.300", "notes"]
for r in raw:
    v = r.replace("[.]", ".").replace("hxxp", "http")
    host = v.split("//")[-1].split("/")[0]
    try:
        ipaddress.ip_address(host); kind = "ipv4"
    except ValueError:
        kind = "domain" if "." in host and not host.replace(".", "").isdigit() else "invalid"
    print(f"{r} -> {host} [{kind}]")

Run with python3 lab_defang.py. Verified output:

hxxps://pay-portal[.]example/login -> pay-portal.example [domain]
198.51.100[.]23 -> 198.51.100.23 [ipv4]
update-svc[.]example -> update-svc.example [domain]
203.0.113.300 -> 203.0.113.300 [invalid]
notes -> notes [invalid]

3.2 Reading the result

The defanged URL becomes host pay-portal.example, classified as a domain. 198.51.100[.]23 becomes a valid IPv4 address. update-svc[.]example is a domain. 203.0.113.300 is invalid because 300 exceeds the maximum octet value of 255; the ipaddress module rejects it. "notes" is a stray word and is marked invalid rather than guessed.

Two of the five inputs are rejected, and for different reasons. The first is a well-formed address with an impossible octet, caught by the standard library rather than by a hand-written check. The second is not an indicator at all, the kind of stray word that slips into a pasted list. Marking both invalid means a human looks at them instead of the script deciding.

3.3 Text only

The script never opens a connection. Refanging for matching is safe; refanging for clicking is not. Keep defanged forms in documents and refanged forms only inside matching code.

In the lab the refanged value lives only in the variable v and in the printed line, which is a text label, not a link. If a report needs to show the indicator, it shows the defanged form. That way a reader who clicks by mistake reaches nothing, and the matching code still works on the normal form it needs.

3.4 Structure

Small functions — refang, extract host, classify — are easier to test than one long block. Each has one job.

The lab is short enough to keep in one loop, but its three steps are already distinct: refang the value, take the host from a URL, then classify the host. Splitting them into functions lets Lena test each step alone, for example checking that host extraction handles a URL with a path without also testing classification. Small functions also make reviews easier, because each change a colleague reads touches one clearly named job.

3.5 Testing edges

Test with values like trailing dots, capital letters and ports ("host.example:8443"). Each edge case either passes cleanly or produces a clear "invalid" so a human can look.

Each of those edge cases carries a clear risk for matching. A trailing dot or capital letters could make the same domain look like two values, and a port number could stop the classifier recognising a host. Lena adds such cases to her test list one at a time and writes down the expected label before running the script, so a surprise is visible.

3.6 Comments and names

Name variables for what they hold — host, kind — not x and y. A one-line docstring at the top states the script's purpose and that it uses synthetic data, which helps reviewers and future maintainers.

The lab's own docstring follows this rule: it says what the lab does, that it is synthetic and that it uses no network. A reviewer reading only that line knows the script is safe to run. Clear names matter when the script is reused months later by a colleague who has not seen Lena's notes.

Version control

Keep scripts in the team's repository with short commit messages. When a script's output changes, the history shows which edit caused it, and colleagues can review changes before they affect daily matching.

Worked example

Lena pastes 30 values from an MCX email. Her script classifies 27 and marks three invalid: two had octets above 255, typos introduced when the sender typed them. She asks MCX for corrections instead of guessing which digit was wrong.

Synthetic case. The values are invented.

Errors and limits

Simple host extraction fails for unusual URL forms; keep a manual review queue.

Exercises

Synthesis

Refang only as text for matching, validate with the standard library, flag rather than guess, and test edge cases.

Continue studying — chapter 03 of 30

← Chapter 02 (draft) · Course chapter index · Chapter 04 (draft) →