Cloud Architecture
Advanced 16 min read Cloud Architecture

Event-Driven & Asynchronous Architecture

Let the work choreograph itself

Decoupling via events: pub/sub topologies, event sourcing and outbox patterns, exactly-once-vs-idempotent delivery, dead-lettering, and the 'what do I do while the event does its thing' async mental model. Events make systems scale + stay consistent — with discipline.

event drivenpub/suboutboxevent sourcingidempotencyDLQasyncCQRS
Prerequisites: Data-Driven Architecture Serverless
Before we start

What you will learn

  • Model async flows with queues/events
  • Implement idempotent consumers + DLQ
  • Understand outbox/event-sourcing for consistency
In 60 seconds

Events decouple producers from consumers: producers publish, consumers react, queues/streams hold the resume point. The hard parts: delivery semantics (at-least-once → idempotent consumers), ordering, the outbox pattern (commit-then-publish atomically), and DLQs for poison messages.

Introduction

What is this?

Mechanics: queues (SQS/Redis/Q Azure, Kafka/streams), pub/sub (SNS/EventBridge/Pub-Sub), topics/partitions (Kafka for ordering+replay), dead-letter queues. Patterns: event-driven (reactions), event sourcing (state is the event log; projection = read model), CQRS (write model separate from read model), outbox (transactional: write event to DB table in same txn, a publisher forwards → avoids publish-lost), saga (long workflows, per-step compensation), idempotent consumer (event keys, replay-safe). Delivery: at-least-once (default), exactly-once (best-effort, via idempotency + dedupe), ordering (single-partition/key). Testing: replay events to rebuild state, DLQ inspection, poison-message handling.

Motivation

Why do you need it?

  • Async decoupling lets each component scale/time/evolve independently — the growth lever most monoliths lack.
  • Event sourcing gives you an auditable history and a free rebuild path (projections).
  • Outbox gets you 'commit then publish' without the two-phase commit class of bugs.
  • Idempotent consumers make retries safe — the enabler of the whole 'at-least-once' world.
Think about it

Mental model

Imagine

A kitchen with sous-chefs who put prep-notes on a clipboard; each station works on its own schedule

Think of

Event-driven kitchen

The clipboard (queue of tickets) decouples order-taking from plating: prep doesn't wait for the head chef's cadence. Distribution (topic) takes tickets to the right stations. When a station is down, tickets wait on the board (DLQ→retry) — no dependency stalls the line.

Mechanics

How it works

  • Producers publish domain events (order.created) to topics/queues.
  • Consumers subscribe; each has its idempotency key + DLQ.
  • Outbox: write the event in the same transaction as the state change; a relay publishes it (no lost event).
  • Consumers project state (event source) or react (transient); retries/DLQ handle poison.
  • Sagas coordinate long flows by compensation, not by synch Wait-on-all.

Event flow.

Write path
Relay
Consumers
Projections
Replay
Hands-on

Examples

Outbox + idempotent consumer

Outbox + idempotent consumer
# 1. In the DB transaction that saves Order:
#    INSERT order ...; INSERT outbox(id, order_id, event_type, payload)
# 2. Relay: poll/table → publish event to topic; mark delivered
#    (if relay dies mid-way, idempotent re-publish ok: consumer dedupes)
# 3. Consumer(order.created): 
#    if not exists(processed, id='order-123'): notify + do work
# 4. On consumer crash mid-work: DLQ → inspect → requeue.

The outbox makes 'publish' atomic with the write; idempotent consumers make the at-least-once delivery safe. That's the whole trick of the pattern.

Real world

In production

  • Order service: write txn → outbox → SNS/SQS consumers for email, warehouse, billing.
  • CQRS: write model (relational) + read projections (cache/NoSQL) fed by events.
  • Saga for checkout: reserve stock → charge → ship, with compensate steps on any failure.
Guidelines

Best practices

  • Encode idempotency: event key/id + dedupe store in every consumer.
  • DLQ + poison-message runbook for every queue; inspect and requeue safely.
  • Use outbox for anything transactional; never publish 'in a callback after commit' (loss window).
  • Ordering when it matters: single partition per key (Kafka) or SQS FIFO groups.
  • Replay is a feature: events must be replayable to rebuild state (that's your DR superpower).
Watch out

Common mistakes

  • Publishing events outside the txn → the classic 'order saved, notification lost'.
  • Consumers without idempotency → retry double-applies (cancel then re-cancel).
  • No DLQ for poison events → one bad payload wedges the whole queue.
  • Cross-service transactions (synchronizing state via locks) instead of sagas/outbox.
When things break

Debugging tips

  • Events silent → check relay/outbox pipeline + consumer offset; a dead consumer without lag alerting.
  • Poison event stalls the queue → DLQ inspection first; fix the producer schema or the consumer parsing, then requeue.
  • Duplicated effects despite 'exactly once' → idempotency key missing/reused; add key + dedupe store.
  • Ordering chaos → routed by hash(key) to wrong partition; force ordering by partition-key on the topic.
Interviews

Interview checkpoint

Practice

Hands-on lab

Stand an Outbox + Idempotent Consumer
40 min · advanced
In progress
Objectives
  • Model order write + outbox in a small DB
  • Publish + consume with a dedupe check
  • Prove replay safety with a retried event

Stand an Outbox + Idempotent Consumer

40 min · advanced

0/3 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

The pattern that removes the 'event lost between commit and publish' flaw:

2

At-least-once delivery becomes safe when:

Go deeper

Official documentation

AWS event-driven patterns
docs
Outbox pattern (microsoft)
docs
Kafka docs
docs