Logging & Audit
The witness that never lies (if configured)
Audit logs are the witness of your cloud: control-plane actions, object access, policy changes, and key uses. Centralizing, immutability, retention, and alerting turn logs from 'stored' into 'forensically useful'.
What you will learn
- Enable org-wide control-plane audit trails
- Centralize + protect (immutability, retention)
- Wire suspicious-action alerts from audit streams
CloudTrail / Azure activity / GCP audit capture control-plane events; data-plane audit (S3 server access, VPC flow logs) adds depth; SIEM (Splunk/Loki+alt) aggregates. Immutable buckets + alarms make a tamper-evident timeline.
What is this?
Sources: CloudTrail (management+data events, organization trail), Azure activity log + resource logs + NSG flows, GCP Cloud Audit Logs (admin/access/data). Data-plane logs: S3 access logs, CloudWatch/stackdriver/Log Analytique, VPC flow logs (netflow), CloudFront realtime, DNS logs. Delivery: to SIEM bucket + SIEM, plus per-service log sinks. Protection: S3 object lock (legal hold/retention), WORM, deny-destroy policies, keyed at-rest encryption. Retention: configurable (90d-2555d cloudtrail; archival to S3/glacier). Alerting: EventBridge/cloudwatch rules on sensitive actions (CreateKey, PutBucketPolicy, DeleteUser, org change), CL= activity explorer; anomaly dashboards. Multi-account: org-level trails + central bucket role-scoped.
Why do you need it?
- Audit logs answer the #1 incident question: 'what changed and who did it'.
- In-place immutable storage stops an attacker from erasing their own tracks.
- VPC flow logs expose lateral movement invisible in app logs.
- Alert-on-sensitive-actions is the difference between a blip and a breach.
Mental model
A casino's camera + tip ledger record every chip movement and are kept sealed
Cloud audit logs
Like the pit boss, you review the sealed tapes (immutable audit) when something goes wrong — the tape doesn't forget, and it can't be edited mid-argument.
How it works
- Provider writes events to the audit log on every authorized action.
- CloudTrail delivers to a central S3 bucket per region/trail (org-wide).
- Retention policy locks the object (object lock/bucket policy) for the required window.
- SIEM ingests, indexes, and provides search/dashboards; alert rules fire on patterns.
- Immutability verified by periodic object-lock/audit check (or SIEM hashing) to defeat tampering.
Audit pipeline.
Examples
Sensitive-action alert rule
# EventBridge rule (AWS)event pattern: { source: ["aws.kms","aws.s3","aws.iam"], detail-type: ["AWS API Call via CloudTrail"], detail: { eventName: ["CreateKey","PutBucketPolicy","DeleteUser","DetachPolicy"], "readOnly": [false] }}# → route to sec-ops SNS + SIEMThe rule is declarative: destructive/policy actions from any identity funnel into your incident channel.
In production
- Org CloudTrail + central bucket with object lock → SIEM custom query playbook.
- VPC flow logs for all subnets → detect lateral moves / exfil paths.
- Automated resource-destroy log alert → page on-call within a minute.
Best practices
- Enable org-wide trails + data-plane logs in every region on day one.
- Store audit orthogonally to app logs: SIEM or sec-bucket, not the app-bucket.
- Immutable retention (object lock) sized to compliance + forensic need.
- Write alert rules for policy/destroy/create-sensitive + anomaly-detection on top.
- Review alert hygiene quarterly (false positives drown signal).
Common mistakes
- Trail only in one region, so cross-region actions unlogged.
- Audit bucket same as app bucket with public policy → logs leak history.
- Retention 'forever' at the wrong layer (SIEM truncates while shed keeps nothing).
- No alert on the audit trail itself being deleted or reconfigured.
Debugging tips
- Missing events → trail/region coverage of resource type and data-vs-management event toggle.
- Suspicious gaps in timeline → check object lock/sink delivery lag; VPC flow + trail cross-ref.
- SIEM noise → broaden alert-to-respond thresholds by severity and add silent-window exclusions.
- Attackers taming their tracks → immutable-all: can't un-log immutable buckets.
Interview checkpoint
Quick quiz
Which log type captures network-level flows between resources?
The immutability weapon is: