Topic 11 of 12
Observability and incident response
Learn to see inside a running system with metrics, logs and traces, set alerts and SLOs, and respond to incidents with a local Prometheus and Grafana lab.
After this topic you can: You can instrument a small service, query and chart its metrics, alert on user-facing symptoms, and run a calm incident from detection to a written postmortem.
8 lessons · 92 min · 0 of 8 done
Best read after: Containers with Docker, Kubernetes
Draft. This lesson is written and waiting for a person to run every command and review it. It is safe to learn from, but check the official documentation for anything you depend on.
Lessons, in order
- 1Monitoring vs observability: metrics, logs, tracesCoreLearn what monitoring and observability mean, what metrics, logs and traces each tell you, and see all three from a tiny local app.10 min
- 2Metrics with Prometheus: scraping and PromQLCoreUnderstand how Prometheus pulls metrics from a /metrics page, how counters differ from gauges, and run your first PromQL queries.14 min
- 3Logs that help: structure, levels, correlation IDsCoreLearn how structured logs, levels and a request ID turn a pile of text into something you can search, and where Loki fits in.9 min
- 4Traces and OpenTelemetry: following one requestAdvancedUnderstand spans, traces and context propagation, what OpenTelemetry is, and generate your own spans locally with no backend.12 min
- 5Dashboards with Grafana: charts that answer questionsCoreConnect Grafana to Prometheus and build a small dashboard around traffic, errors, latency and saturation that you can read in seconds.10 min
- 6Alerting and SLOs: paging on what users feelCoreLearn SLIs, SLOs and error budgets as Google's SRE material defines them, then write a Prometheus alert on a symptom and watch it fire.15 min
- 7Incident response: triage, communicate, mitigateCoreLearn the roles and order of work in an incident, then practise detect, triage, mitigate and log a timeline on a deliberately broken lab app.13 min
- 8Blameless postmortems: learning after the incidentCoreLearn what a good postmortem contains, why blameless wording works, and write one from your lab incident using a simple template.9 min