SOC Operations In progress
Intermediate 17 min read SOC Operations
Alert Triage That Scales
The discipline of deciding fast and honestly: triage criteria, enrichment, the deny-of-action before escalation, and the queue hygiene that keeps analysts sane.
triageenrichmentfalse positiveescalationplaybookqueue hygiene
Introduction
What is this?
Triage is the decision process that turns every alert into a verdict — true sign, benign, or escalate — with enough evidence that the next tier trusts the conclusion.
Motivation
Why do you need it?
- Most alerts are benign; triage's value is systematically proving it so true positives get real attention.
- A queue without standards produces escalation roulette — every ticket is a coin flip of response depth.
- Good triage writes the analyst's instinct down as articulated logic, reviewable and improving.
Think about it
Mental model
Imagine
An emergency room intake
Think of
The department triage nurse
Every patient is assessed, prioritized, and routed — the nurse's judgment is the difference between calm flow and chaos.
Mechanics
How it works
- Each alert gets a verdict: true/benign/unknown, with the 'why' in one or two lines.
- Enrichment adds context: user, asset, previous activity, whitelist status, ATT&CK cell.
- Escalation follows found evidence: scoped containment steps per playbook, not improvisation.
- Queue hygiene: metrics on age, MTTA/MTTR, and false-positive rate drive rule tuning.
One alert's triage path
Alertlands in the queue
Enrichuser, asset, history, rule intent
Verdicttrue / benign / unknown
Actioncontain, close, or escalate
Feed backrule tuning from the verdict
Real world
In production
- A SOC's triage doc says: 'benign if a whitelisted deploy host and matching change ticket'; analysts click through 60% of alerts in seconds.
- A team's escalation playbook names exact containment steps, so tier 2 sees a ticket with a plan, not a prayer.
- An org reports MTTA weekly; triage time drops after each tuning cycle, visibly.
Guidelines
Best practices
- Write the triage checklist per alert family: what evidence makes it benign, what makes it real.
- Require a verdict + reason per ticket; 'looked fine' is not a verdict.
- Use playbooks for escalation: who, what step, how long, and who to notify.
- Track and publicize queue stats; visibility keeps the process honest.
Watch out
Common mistakes
- Escalating everything 'to be safe', flooding tier 2 until real incidents drown.
- Closing an alert without a reason, so the tuning loop learns nothing.
- Triage by memory of 'the last time we saw this' without checking the actual fields.
- Letting the queue age silently until SLA misses become the norm.
When things break
Debugging tips
- When verdicts disagree, open the raw event and compare with the rule's selector — the mismatch is the insight.
- Track 'benign because' reasons weekly; the top reason is a tuning candidate.
- Re-triage a historical spike in a scratch dashboard to sharpen the process before the next wave.
Security
Security considerations
- Verdicts are judgment calls with audit trail value; record them, not just the outcome.
- Escalation decisions involve data exposure care: annotate evidence, never delete or edit the raw.
Interviews
Interview checkpoint
Practice
Hands-on lab
Run a Triage Drill on a Synthetic Alert Batch
35 min · intermediate
Objectives
- Generate a batch of synthetic alerts with mixed verdicts.
- Apply a written triage checklist in a scripted decision.
- Produce a weekly-style verdict table with reasons.
Run a Triage Drill on a Synthetic Alert Batch
35 min · intermediate
0/3 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself
Quick quiz
0/2
1
The correct response to an 'unknown' verdict is...
2
Escalating every alert...
Go deeper