Chapter 13 — Troubleshooting conflicting alerts and missing telemetry
Lesson map
Lesson 13 published. Map, example, EX049.
flowchart TD
N0["Learning outcomes and connection"]
N1["13.1 Translate prose back into predicates"]
N2["13.2 An offline predicate check"]
N3["13.3 Diagnose disagreement with a coverage contract"]
N4["13.4 A versioned escalation"]
N5["Subject synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5
N5 --> EX["Worked example"]
EX --> Q["Exercise EX049"]
Logical drawing
flowchart LR
N0["Learning outcomes and connection"]
N1["13.1 Translate prose back into predicates"]
N2["13.2 An offline predicate check"]
N3["13.3 Diagnose disagreement with a coverage contract"]
N4["13.4 A versioned escalation"]
N5["Subject synthesis"]
N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> Q["EX049"]
[Learning outcomes and connection]
|
v
[13.1 Translate prose back into predicates]
|
v
[13.2 An offline predicate check]
|
v
[13.3 Diagnose disagreement with a coverage contract]
|
v
[13.4 A versioned escalation]
|
v
[Subject synthesis]
|
v
[Worked example] --> [Exercise EX049]
Learning outcomes and connection
The learner will reconstruct two alert predicates from the K12 accepted records, identify where their claims exceed their inputs, treat missing telemetry as a coverage question, and write a versioned escalation that does not choose an actor prematurely. Chapter 12 produced four accepted and two rejected rows; Chapter 13 asks what a detection rule may safely say about them. Chapter 14 will automate part of this triage while retaining an auditable exception path. NIST SP 800-61 Rev. 3 describes incident response as part of risk management, and NIST SP 800-92 discusses log management at a high level. Neither publication validates the invented K13 alerts or prescribes this classroom's rules.
13.1 Translate prose back into predicates
The synthetic monitor generates AL13-A: “svc-book reached the protected reservation service” because accepted app A1 has principal svc-book, session s-7, request r-601 and kind queued. Another monitor generates AL13-B: “svc-book was blocked” because accepted idp I2 has the same principal label, a different session s-8, and kind denied. The dramatic wording suggests disagreement. The actual predicates describe different events. AL13-A at most reports that a supplied app row says a task was queued. AL13-B at most reports one IdP denial for s-8. Both can be true; neither establishes a database commit, a universal block, a human actor or compromise.
Before resolving an alert, collect its rule version, query/filter interval, producer and object IDs, matching row IDs, join keys, clock basis, ingestion lag and known exclusions. Ask which data source could falsify the rule's conclusion. A rule that groups only by principal may merge many sessions. A rule that says “blocked” from one denied session may hide simultaneous success in another session. A rule that says “reached” from an HTTP 202 or queued event may overstate the business outcome. A good detector makes its scope visible: “A1 reports an accepted queue entry for r-601 under a claimed s-7 field,” not “the attacker reserved a room.”
The K13 alert packet is derived from K12; it is not an extra main-case release. K12's I1 reports s-7 success and I2 reports s-8 denial. Even the apparent I1–A1 s-7 link needs producer-validated namespace and completeness. The app A2 row for r-602 failed time validation and did not enter the accepted view. An alert query over only accepted rows could therefore report “no r-602 event” despite the raw object containing a structurally unusable r-602 line. The rejection report is part of the evidence for a telemetry-quality concern, not evidence that the request occurred as stated.
13.2 An offline predicate check
The code below models the two alert triggers and one proposed inference. It uses literal already accepted K12 fields. It does not connect to a SIEM or decide whether any event is authentic. Notice that the labels are chosen to describe what the Boolean actually checks. Run it with Python 3 or reason through the expressions by hand.
idp = [
{"id": "I1", "principal": "svc-book", "session": "s-7", "kind": "success"},
{"id": "I2", "principal": "svc-book", "session": "s-8", "kind": "denied"},
]
app = {"id": "A1", "principal": "svc-book",
"session": "s-7", "request": "r-601", "kind": "queued"}
print("app_queue_claim:", app["kind"] == "queued")
print("idp_denial_claim:", any(row["kind"] == "denied" for row in idp))
print("denial_matches_app_session:",
any(row["kind"] == "denied" and row["session"] == app["session"]
for row in idp))
print("db_commit_in_supplied_rows:", False)
The third line is False because the denied I2 session is s-8, while A1 states s-7. This is a literal string comparison, not a validated cross-producer join. The fourth line is False because no database commit record is supplied; it is not evidence that the database did not commit. The wording “in supplied rows” matters. Change I2's session to s-7 in a separate variant and the third Boolean changes; that would make the alerts concern the same literal session label, but a source-mapping and timing review would still be needed before asserting one transaction or cause.
13.3 Diagnose disagreement with a coverage contract
Treat detection as a pipeline: producer emission → transport → ingestion → parsing → indexing → rule query → alert rendering. A missing alert can result from failure or filtering at any stage, or from no event. An extra alert can result from duplicate ingestion, broad joins, stale rule state or a legitimate event whose meaning was overinterpreted. For each stage, ask for a permitted observable: producer counters and schema version; transport delivery/error counts; ingestion lag and dropped-event counters; parser accepted/rejected counts; index freshness; query filters and access permissions; rule revision and matched row IDs. A dashboard's green status is useful only for the path and interval it actually tests.
For K12 app data, the intake report says three raw app rows, one accepted and two rejected. That immediately limits any alert over accepted app rows. A source owner could provide a known, in-range control event for the same app route that should appear under the rule's filter. If absent, a negative result for r-602 is especially weak. If present, confidence increases for that route and interval, but does not certify all routes or repair A2's malformed time. An out-of-range control should not appear in the narrow query; if it does, the filter is suspect. The control's producer receipt must be independent of the analyst's spreadsheet. These are proposed tests under owner approval, not actions AUTH-K12 permits on real telemetry.
Use three-valued language for a decision: supported within this packet, contradicted within this packet, or unknown because the relevant source/coverage is missing. “No confirmed database commit record in supplied rows” is supported. “The database did not commit” is unknown. “The same IdP session was both denied and queued” is contradicted by the literal IDs s-8 and s-7, but producer namespace validation remains open. This structure keeps a learner from equating missing telemetry with a negative finding.
13.4 A versioned escalation
A defensible version 1 note at the 12:06 workshop review would state: “AL13-A was triggered by the supplied app A1 queued row under s-7; AL13-B by the supplied IdP I2 denied row under s-8. The principal labels match, but the session fields differ. K12's app parser rejected two of three app rows, including r-602's malformed time. No database commit receipt or verified cross-source session mapping was supplied. The discrepancy may reflect ordinary parallel sessions, incomplete telemetry or a rule that overstates its predicate. The incident lead should request owner-approved session namespace definitions, app queue-to-commit receipts and source-health counters for the bounded window. A principal-wide revocation or announcement of compromise is not supported by this packet.” This is a classroom example, not a real incident instruction.
If a validated producer mapping later proves A1 and I1 belong to the same s-7 namespace, version 2 can strengthen the session correlation while leaving the database outcome unknown. If a corrected app export validates an r-602 row, version 2 should add a new raw-object receipt, parser result and changed alert evaluation; it must not erase K12 v1's rejection. If a source owner reports missing ingest coverage, the correct next step is a time-bounded preservation and coverage investigation, not a guessed attacker story. A response owner may still take precautionary action based on broader evidence and risk, but that decision must be recorded separately from what these rows alone prove.
Common errors and limits
- Treating alert titles as data rather than reading their rule predicates and matched records.
- Joining two sessions by principal label or nearby time.
- Calling an app queue record a database commit or a patron-visible success.
- Equating a missing accepted row with no producer event while the raw rejection report exists.
- Treating a green ingestion dashboard as proof that the precise route, filter and time window were covered.
Exercises — subject 13
EX049 — Alert predicate reconstruction. Objective: reduce AL13-A/B to testable claims. For each alert, state exact matched source row, predicate, wording that is too broad, and a corrected title. Include the relationship between s-7 and s-8. Evidence: two-row table. Rubric: 2 = different sessions and queue/denial scopes explicit; 1 = notes conflict without predicate; 0 = declares an identified attacker.
EX050 — Predicate transcript and variant. Objective: reproduce §13.2. Record four exact output lines, then change only I2's session to s-7 in a copy and state which line changes. Explain why the fourth line is absence of a supplied record rather than proof of no commit. Evidence: original and variant transcript. Rubric: 2 = exact booleans and bounded inference; 1 = variant correct but overstates join; 0 = calls the variant a new main-case finding.
EX051 — Sensor-health experiment. Objective: design a discriminating telemetry check. For the app source and a 12:00:00–12:00:10 query, specify one producer-backed in-range control, one out-of-range control, the three-raw/one-accepted/two-rejected accounting, ingestion lag and rule version. Describe how each control failure changes confidence in “no r-602.” Evidence: coverage card. Rubric: 2 = stages, receipts and both control outcomes; 1 = only a generic health check; 0 = treats missing accepted row as no event.
EX052 — Versioned escalation. Objective: synthesize a restrained decision. Write a 170–230-word version 1 note comparing alerts, stating unknown DB outcome and pending authority; then add a two-sentence version 2 if the custodians validate the s-7 mapping but cannot retrieve a DB receipt. Name one reversible option to consider under a separate approval, its service risk and a falsifiable trigger. Evidence: two-version handoff. Rubric: 2 = evidence, source limits, version change and owner-led decision; 1 = cautious wording without a next test; 0 = orders an unapproved broad control.
Subject synthesis
The K13 alerts report different predicates over a limited and partly rejected record set. They can both be true without settling the reservation outcome. The technical job is to trace each claim through producer, parser, query, rule and business source, then preserve the unresolved branches. Chapter 14 turns these limited inputs into a bounded, reproducible review packet.
Continue studying — chapter 13 of 18
← Chapter 12 · Course chapter index · Chapter 14 →
Continue to the next authored lesson.