Chapter 15 — Web3, blockchain and cryptoasset security: guided lab
Lesson map
Lesson 15 published. Map, example, EX043.
flowchart TD
N0["Learning outcomes"]
N1["15.1 Why synchronisation matters"]
N2["Lab — Node synchronisation check"]
N3["15.2 Interpreting the lag"]
N4["15.3 Responding with a runbook"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX043"]
Logical drawing
flowchart LR
N0["Learning outcomes"]
N1["15.1 Why synchronisation matters"]
N2["Lab — Node synchronisation check"]
N3["15.2 Interpreting the lag"]
N4["15.3 Responding with a runbook"]
N5["Synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX043"]
[Learning outcomes]
|
v
[15.1 Why synchronisation matters]
|
v
[Lab — Node synchronisation check]
|
v
[15.2 Interpreting the lag]
|
v
[15.3 Responding with a runbook]
|
v
[Synthesis]
|
v
[Worked example] --> [Exercise EX043]
Learning outcomes
You will (1) check node synchronisation from height samples; (2) interpret lag; (3) respond with a runbook step.
15.1 Why synchronisation matters
Each node should hold the same chain. If one falls behind, applications reading from it see old balances, and if two fall behind, the ledger may stop producing blocks. Lucas collects block heights from the three nodes every minute and checks whether any node is more than two blocks behind the highest. This lab uses five synthetic samples. It is read-only and uses only Python's standard library. Monitoring synchronisation is one of the most basic and valuable checks a technologist can run, and it often reveals problems before members notice anything.
Lab — Node synchronisation check
"""Flag ledger nodes that lag behind the highest block height (synthetic, read-only)."""
# minute: heights reported by node-a, node-b, node-c
SAMPLES = [
(0, 1200, 1200, 1199), (1, 1206, 1206, 1205), (2, 1212, 1211, 1207),
(3, 1218, 1218, 1209), (4, 1224, 1223, 1210),
]
NAMES = ("node-a", "node-b", "node-c")
MAX_LAG = 2
def main():
for minute, *heights in SAMPLES:
top = max(heights)
for name, h in zip(NAMES, heights):
if top - h > MAX_LAG:
print(f"minute {minute}: {name} lags {top - h} blocks")
print(f"samples {len(SAMPLES)}; threshold {MAX_LAG} blocks")
if __name__ == "__main__":
main()
Run with python3 lab_sync.py. Verified output:
minute 2: node-c lags 5 blocks
minute 3: node-c lags 9 blocks
minute 4: node-c lags 14 blocks
samples 5; threshold 2 blocks
15.2 Interpreting the lag
The lab reports that node-c lags 5 blocks at minute 2, 9 at minute 3 and 14 at minute 4, while node-a and node-b stay within one block of each other. The growing lag suggests node-c has stopped keeping up, rather than suffering a brief delay. Lucas considers likely causes: a full disk, an overloaded processor, lost peer connections or a clock problem. He checks disk use first, because the earlier chapter showed it is the most common cause. In this synthetic case, node-c's disk is at 99 percent, which explains its inability to store new blocks.
15.3 Responding with a runbook
Lucas follows the runbook step for a lagging node: confirm the other two nodes are healthy, point the booking application at a healthy node, free disk space by moving old logs as the runbook describes, restart the node and watch it catch up. Node-c recovers within twenty minutes. He records the event and finds that the 80 percent disk alert had never been enabled on node-c, so he enables it. Ms Adebayo notes that runbooks turn diagnosis into consistent action, so any trained volunteer can respond safely, even late in the evening when no experienced staff member is available.
Worked example
The lab flags node-c lagging 5, 9 and then 14 blocks; a full disk is found, the runbook restores it in twenty minutes and the missing 80 percent disk alert is enabled.
Synthetic case. The heights, disk usage and recovery are synthetic.
Errors and limits
Treating a growing lag as a brief delay lets a failing node drift further behind.
Exercises
- EX043 — Lag threshold. What lag threshold does the lab use? Rubric: 3 = two blocks;; 2 = 2 blocks;; 1 = ten blocks;; 0 = none.
- EX044 — Lagging node. Which node lags, and by how much at minute 4? Rubric: 3 = node-c, 14 blocks;; 2 = node-c;; 1 = node-a;; 0 = none.
- EX045 — Root cause found. What caused node-c's lag? Rubric: 3 = its disk was at 99 percent;; 2 = full disk;; 1 = network;; 0 = none.
Synthesis
Simple height monitoring and a runbook catch and fix lagging nodes quickly.
Continue studying — chapter 15 of 30
← Chapter 14 (draft) · Course chapter index · Chapter 16 (draft) →