◇ OpenCyber Academy
Technical · Chapter 8 · English

Chapter 8 — Network and proxy logs

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 8 — Network and proxy logs

Lesson map

Lesson 08 published. Map, example, EX029.

flowchart TD
  N0["Learning outcomes"]
  N1["8.1 Proxy logs"]
  N2["Lab — matching indicators"]
  N3["8.2 Lab result"]
  N4["8.3 The second host"]
  N5["8.4 Firewall logs"]
  N6["8.5 Encryption"]
  N7["8.6 Indicators expire"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7
  N7 --> EX["Worked example"]
  EX --> Q["Exercise EX029"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["8.1 Proxy logs"]
  N2["Lab — matching indicators"]
  N3["8.2 Lab result"]
  N4["8.3 The second host"]
  N5["8.4 Firewall logs"]
  N6["8.5 Encryption"]
  N7["8.6 Indicators expire"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> N7 --> Q["EX029"]

[Learning outcomes]
    |
    v
[8.1 Proxy logs]
    |
    v
[Lab — matching indicators]
    |
    v
[8.2 Lab result]
    |
    v
[8.3 The second host]
    |
    v
[8.4 Firewall logs]
    |
    v
[8.5 Encryption]
    |
    v
[8.6 Indicators expire]
    |
    v
[Worked example] --> [Exercise EX029]

Learning outcomes

You will (1) read web proxy logs; (2) match traffic against indicators; (3) identify which hosts need checking; (4) understand limits of network data.

8.1 Proxy logs

A web proxy records which internal host visited which domain and when. It is ideal for answering "who clicked?". In the finance case, the proxy log showed that only one of thirteen recipients, the finance PC at 10.2.0.31, visited invoice-portal.example, and it did so twice. That narrowed the password reset to one user rather than all thirteen recipients. The lab below repeats this search on a small synthetic sample with a list of indicators. The time window for that search runs from the email's arrival to its removal.

Lab — matching indicators

"""Match synthetic proxy-log domains against a synthetic indicator list."""
iocs = {"invoice-portal.example", "update-check.example", "cdn-files.example"}
proxy = [("10.2.0.14", "council.example"), ("10.2.0.31", "invoice-portal.example"), ("10.2.0.31", "invoice-portal.example"),
         ("10.2.0.52", "news.example"), ("10.2.0.77", "cdn-files.example"), ("10.2.0.14", "mail.example")]
hits = {}
for host, dom in proxy:
    if dom in iocs:
        hits.setdefault(host, set()).add(dom)
for host in sorted(hits):
    print(host, sorted(hits[host]))
print("hosts to check:", len(hits), "of", len({h for h, _ in proxy}))

Run with python3 lab_ioc.py. Verified output:

10.2.0.31 ['invoice-portal.example']
10.2.0.77 ['cdn-files.example']
hosts to check: 2 of 4

8.2 Lab result

Of four hosts in the sample, two contacted domains on the indicator list: 10.2.0.31 visited invoice-portal.example twice (the finance PC from chapter 7), and 10.2.0.77 visited cdn-files.example. The second hit came from an indicator unrelated to the phishing email, which is why it needed its own check. 10.2.0.77 matched cdn-files.example, which came from the national alert, not the email. Carlos therefore treats it as a separate question and opens its own case rather than adding it to the phishing incident. The lab counts four distinct hosts in the sample, so two of four needed checking.

8.3 The second host

10.2.0.77 was not in the phishing case. Carlos checks: it is a planning-office PC; cdn-files.example came from a separate list of indicators from a national alert. The visit was a single download blocked by the proxy. Opening a separate case keeps the finance incident record clean and lets each case close on its own evidence. The block at the proxy meant the file never arrived, and the endpoint agent showing no new processes confirmed nothing ran. The national alert was the source of the indicator, rated reliable.

8.4 Firewall logs

Firewalls record allowed and blocked connections by address and port; they help when traffic bypasses the proxy. If an attacker's tool connected to an address directly, bypassing the proxy, only the firewall would record it. In Northgate's cases the proxy answered most questions, but the firewall log is the check that the finance PC made no other unexpected outbound connections after its two visits to invoice-portal.example. Carlos checks the firewall logs for the same time window as the proxy search, from the email's arrival to its removal.

8.5 Encryption

Most web traffic is encrypted; proxies see domains but often not full addresses or content. For the finance case, seeing the domain was enough: invoice-portal.example appeared twice for 10.2.0.31. The proxy could not show what the officer typed on the page, so confirming the password entry needed her own account. When content matters, other sources such as the endpoint agent must fill the gap. The proxy's limits are written into the case note so readers do not assume it saw more than domains.

8.6 Indicators expire

Domains on indicator lists get reused or cleaned up; old indicators cause false alarms. invoice-portal.example may be abandoned or taken over by someone else within months. An old indicator then flags innocent visits and wastes analysts' time, which is why Northgate sets review dates on indicators and drops those no longer reported. The blog list rated unknown is never used for automatic blocking. Each indicator in Northgate's list carries the date it was added and its source rating, so stale entries are easy to find.

Time windows

Carlos searches the proxy logs from the time the phishing email arrived until it was removed. Visits before arrival cannot be linked to the email, and visits after removal suggest another source of the link, such as a forwarded copy on a personal device. Bounding the search by time keeps the result small and relevant, and it makes the reasoning easy to explain to reviewers.

Recording the search

Carlos saves the exact proxy query in the case so others can repeat it.

Worked example

Carlos opens a second, low-priority case for 10.2.0.77, confirms the endpoint agent shows no new processes after the blocked download, and closes it with evidence. The finance case is linked to the phishing incident.

Synthetic case. Hosts and domains are synthetic and use reserved example names.

Errors and limits

Matching indicators finds known bad things only; new domains will not match.

Exercises

Synthesis

Proxy and firewall logs show who went where; indicators help but expire and miss new threats.

Continue studying — chapter 08 of 18

← Chapter 07 (draft) · Course chapter index · Chapter 09 (draft) →