Chapter 21 — Log monitoring
Lesson map
Lesson 21 published. Map, example, EX061.
flowchart TD
N0["Learning outcomes"]
N1["21.1 The question"]
N2["21.2 The log lab"]
N3["Lab — Parsing a synthetic authentication log and ran"]
N4["21.3 What it means"]
N5["21.4 Acting"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX061"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["21.1 The question"]
N2["21.2 The log lab"]
N3["Lab — Parsing a synthetic authentication log and ran"]
N4["21.3 What it means"]
N5["21.4 Acting"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX061"]
[Learning outcomes]
|
v
[21.1 The question]
|
v
[21.2 The log lab]
|
v
[Lab — Parsing a synthetic authentication log and ran]
|
v
[21.3 What it means]
|
v
[21.4 Acting]
|
v
[Worked example] --> [Exercise EX061]
Learning outcomes
You will (1) parse authentication logs with scripts; (2) separate internal mistakes from external attacks; (3) act on what the logs show.
21.1 The question
In his third year, Jamal is asked to look at authentication logs from Riverbend's file transfer server, which the booking supplier uses to send nightly reports. The IT manager has noticed the server running slowly. Jamal decides to count failed logins by source address, because a large number of failures from a few addresses usually means password guessing. Ms. Grant reminds him to check that he is authorised to read the logs and to note his question before looking, so he does not go hunting aimlessly.
21.2 The log lab
The lab parses a synthetic log of 3,271 lines with a regular expression and finds 838 failed logins. The top three sources are 203.0.113.45 with 412 failures, 198.51.100.7 with 198 and 203.0.113.88 with 61. All three are external addresses, together responsible for 671 failures, 80.1 percent of the total. The remaining failures come from internal staff addresses and look like ordinary mistyping spread across many accounts. The external failures target names such as root, admin and test. Jamal notes that the addresses belong to ranges reserved for documentation, used here as placeholders for real external addresses, and that a real analysis would check address ownership before drawing conclusions.
Lab — Parsing a synthetic authentication log and ranking sources of failed logins
"""Parse a synthetic SSH-style authentication log and rank sources of failed logins (stdlib only)."""
import random, re
from collections import Counter
random.seed(77)
lines = []
for _ in range(2600):
ip = f"10.4.{random.randint(1,3)}.{random.randint(2,60)}"
ok = random.random() < 0.93
lines.append(f"sshd: {'Accepted' if ok else 'Failed'} password for staff{random.randint(1,40)} from {ip}")
for ip, n in [("203.0.113.45", 412), ("198.51.100.7", 198), ("203.0.113.88", 61)]:
for _ in range(n):
lines.append(f"sshd: Failed password for {random.choice(['root','admin','test','staff3'])} from {ip}")
random.shuffle(lines)
pat = re.compile(r"(Accepted|Failed) password for (\S+) from (\S+)")
fails = Counter(); total_fail = 0
for l in lines:
m = pat.search(l)
if m and m.group(1) == "Failed":
total_fail += 1; fails[m.group(3)] += 1
ext = {ip: n for ip, n in fails.items() if not ip.startswith("10.")}
print(f"lines {len(lines)}; failed logins {total_fail}")
print(f"top sources: {fails.most_common(3)}")
print(f"external sources {len(ext)} with {sum(ext.values())} failures ({sum(ext.values())/total_fail:.1%} of all failures)")
Run with python3 lab_authlog.py. Verified output:
lines 3271; failed logins 838
top sources: [('203.0.113.45', 412), ('198.51.100.7', 198), ('203.0.113.88', 61)]
external sources 3 with 671 failures (80.1% of all failures)
21.3 What it means
The results show that the file transfer server accepts password logins from the internet, and automated tools are guessing common account names. Jamal checks whether any external login succeeded and finds none in the log. He reports to the IT manager that the server is exposed more widely than needed: only the booking supplier should connect, from known addresses. Ms. Grant notes that log analysis often reveals configuration problems rather than successful attacks. He records the finding and the check for successful logins in his notebook.
21.4 Acting
The IT manager restricts the server to the supplier's addresses at the firewall and switches the supplier's account from password to key-based authentication. Jamal reruns his script on the following week's logs and finds external failures have stopped. The server performance returns to normal. He adds the script to a weekly schedule with an alert if external failures appear again. He learns that monitoring is most useful when it leads to a change and then confirms the change worked. The IT manager thanks him.
Worked example
Jamal parses 3,271 synthetic log lines, finds 838 failures with 80.1 percent from three external addresses, finds no successful external login, restricts the server and confirms the fix.
Synthetic case. Logs and addresses are synthetic documentation addresses; outputs come from lab_authlog.py.
Errors and limits
Ranking failures without separating internal and external sources mixes mistyping with attacks.
Exercises
- EX061 — Failed count. How many failed logins does the log contain? Rubric: 3 = 838;; 2 = partly correct;; 1 = wrong;; 0 = none.
- EX062 — Top source. Which address has the most failures? Rubric: 3 = 203.0.113.45 with 412;; 2 = partly correct;; 1 = wrong;; 0 = none.
- EX063 — External share. What share of failures came from external sources? Rubric: 3 = 80.1 percent;; 2 = partly correct;; 1 = wrong;; 0 = none.
Continue studying — lesson 21 (chapter 21 of 30)
← Chapter 20 · Course chapter index · Chapter 22 →
Continue to the next authored lesson.