◇ OpenCyber Academy
Doctoral research · Chapter 2 · English

Chapter 2 — Authorized offensive security and red team: research frontier

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 2 — Authorized offensive security and red team: research frontier

Lesson map

Lesson 02 published. Map, example, EX008.

flowchart TD
  N0["Learning outcomes"]
  N1["2.1 Methodology families"]
  N2["2.2 Open problems"]
  N3["2.3 Critique of weak scores"]
  N4["2.4 ATT&CK as vocabulary"]
  N5["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5
  N5 --> EX["Worked example"]
  EX --> Q["Exercise EX008"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["2.1 Methodology families"]
  N2["2.2 Open problems"]
  N3["2.3 Critique of weak scores"]
  N4["2.4 ATT&CK as vocabulary"]
  N5["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX008"]

[Learning outcomes]
    |
    v
[2.1 Methodology families]
    |
    v
[2.2 Open problems]
    |
    v
[2.3 Critique of weak scores]
    |
    v
[2.4 ATT&CK as vocabulary]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX008]

Learning outcomes

You will (1) map the main methodology families and threat-led frameworks; (2) locate open research problems in measurement, ethics and assurance; (3) critique weak scoring schemes; (4) position ATT&CK as a vocabulary, not a completeness proof; and (5) write a frontier note that motivates a doctorate.

2.1 Methodology families

NIST SP 800-115 organises assessment into planning, discovery, attack (carefully bounded), reporting and remediation support. PTES sequences intelligence, threat modelling, vulnerability analysis, exploitation and post-exploitation under explicit scoping. OWASP WSTG provides web-specific test cases. Threat-led frameworks such as TIBER-EU and CBEST emphasise intelligence-led scenarios, purple-team collaboration and regulatory oversight for critical financial systems. Your frontier map should place each family by what it optimises: breadth of technical coverage, realism of adversary emulation, or assurance for supervisors.

Elena's map places NIST SP 800-115 and PTES under technical coverage, WSTG under web cases, and TIBER-EU and CBEST under threat-led assurance. Her doctorate sits with the first group: she measures whether a scoping instrument reduces out-of-scope probes, not whether an intelligence-led scenario finds business-critical weaknesses.

2.2 Open problems

Open problems that suit a doctorate include: how to measure out-of-scope risk reduction; how to validate that a control blocked an action without publishing an exploit; how coverage of ATT&CK techniques relates to residual risk; how human factors (fatigue, approval theatre) affect ROE compliance; and how to compare red-team report quality. Avoid "problems" that are merely "run more tools".

Elena's problem is the first of these: measuring whether machine-checkable predicates reduce out-of-scope probes. A doctorate that only wraps another scanner has no research question, only an engineering task.

2.3 Critique of weak scores

Aggregating security into a single number — as criticised for OSSTMM's rav (arXiv:2010.06377) — hides what failed and who is affected. Prefer structured findings with impact, evidence and verification status. Severity scores can prioritise work inside one engagement; they are not universal security quantities.

"""Score synthetic findings for report priority (no exploit details)."""
# impact 0-3, exploitability 0-3, exposure 0-3  -> priority
findings = [
    ("SYN-001", "default admin password on staging web UI", 3, 3, 2),
    ("SYN-002", "missing security headers on public site", 1, 1, 3),
    ("SYN-003", "outdated TLS on internal admin port", 2, 2, 1),
    ("SYN-004", "verbose error messages disclose paths", 1, 2, 2),
    ("SYN-005", "service account with domain admin in lab AD", 3, 2, 1),
]

def priority(impact, exploitability, exposure):
    score = impact*4 + exploitability*2 + exposure
    if score >= 16: return "critical", score
    if score >= 11: return "high", score
    if score >= 7: return "medium", score
    return "low", score

for fid, title, i, e, x in findings:
    p, s = priority(i, e, x)
    print(f"{fid} priority={p:8} score={s:2} impact={i} exploitability={e} exposure={x} :: {title}")

Expected output:

SYN-001 priority=critical score=20 impact=3 exploitability=3 exposure=2 :: default admin password on staging web UI
SYN-002 priority=medium   score= 9 impact=1 exploitability=1 exposure=3 :: missing security headers on public site
SYN-003 priority=high     score=13 impact=2 exploitability=2 exposure=1 :: outdated TLS on internal admin port
SYN-004 priority=medium   score=10 impact=1 exploitability=2 exposure=2 :: verbose error messages disclose paths
SYN-005 priority=critical score=17 impact=3 exploitability=2 exposure=1 :: service account with domain admin in lab AD

The findings lab scores impact, exploitability and exposure without describing an attack path. SYN-001, a default admin password on the staging web UI, scores 20 and is critical; SYN-002, missing headers, scores 9 and is medium. Severity helps an operator decide what to fix first inside one engagement; it is not a universal security quantity.

2.4 ATT&CK as vocabulary

MITRE ATT&CK is a living knowledge base of adversary tactics and techniques. Use it to label what was attempted and observed, to communicate with blue teams, and to show coverage gaps. Do not claim that covering a percentage of the matrix means a system is safe — coverage is a planning and reporting aid.

"""ATT&CK-style coverage matrix over synthetic authorized techniques (names only)."""
# Synthetic engagement log: technique_id -> observed (True/False)
PLAN = ["T1046", "T1595", "T1082", "T1083", "T1059", "T1078", "T1021"]
OBSERVED = {"T1046", "T1595", "T1082", "T1083", "T1078"}
LABELS = {
    "T1046": "Network Service Discovery",
    "T1595": "Active Scanning",
    "T1082": "System Information Discovery",
    "T1083": "File and Directory Discovery",
    "T1059": "Command and Scripting Interpreter",
    "T1078": "Valid Accounts",
    "T1021": "Remote Services",
}
covered = 0
for tid in PLAN:
    hit = tid in OBSERVED
    covered += hit
    print(f"{tid} {LABELS[tid]:34} {'COVERED' if hit else 'PLANNED-NOT-SEEN'}")
print(f"coverage={covered}/{len(PLAN)} ({100*covered/len(PLAN):.0f}%)")

Expected output:

T1046 Network Service Discovery          COVERED
T1595 Active Scanning                    COVERED
T1082 System Information Discovery       COVERED
T1083 File and Directory Discovery       COVERED
T1059 Command and Scripting Interpreter  PLANNED-NOT-SEEN
T1078 Valid Accounts                     COVERED
T1021 Remote Services                    PLANNED-NOT-SEEN
coverage=5/7 (71%)

The coverage lab shows 5 of 7 planned techniques observed, or 71%. Planned-not-seen techniques such as T1059 and T1021 go on the next authorized plan. Elena uses ATT&CK to label what operators attempted and what was observed, and to talk with blue teams, not to claim residual risk.

2.5 Worked example (synthetic NHUSTL case)

Elena's frontier note: prior work describes ROE in prose; threat-led frameworks require scenario fidelity; few studies measure whether machine-checkable predicates reduce out-of-scope actions. The findings lab shows prioritisation of synthetic issues without exploit detail. The coverage lab shows 71% of a planned technique set observed — a reporting fact, not a safety proof. Her gap: a controlled comparison of prose vs checkable scope on out-of-scope probe rate.

The gap is measurable, ethical and non-tool: a controlled comparison of prose against checkable scope on out-of-scope probe rate, in authorized labs with synthetic targets. That sentence becomes the first paragraph of her proposal.

2.6 Errors and limits

Each error weakens the frontier note. Reading ATT&CK coverage as risk eliminated turns a planning aid into a false claim of safety. A single-score security level hides what failed, as Albrecht and Jensen argue. Listing tools instead of questions produces an engineering project, not a doctorate. Check your frontier note for all three before the proposal. Elena's note avoids all three.

Exercises

Synthesis

The research frontier sits where methodology, measurement and ethics meet — not where another scanner can be wrapped. Map frameworks, reject empty scores and state a measurable gap.

Continue studying — chapter 02 of 08

← Chapter 01 (draft) · Course chapter index · Chapter 03 (draft) →