Serverless
Advanced 15 min read Serverless

Stateful Serverless Patterns

Long jobs and sagas on stateless compute

Functions are stateless; durable workflows (Step Functions, Logic Apps, Workflows) bring state as orchestrated sequences with retries, human approval, and exact progress tracking. Long-running, resumable, auditable serverless — built from stateless bricks.

Step FunctionsDurable FunctionsWorkflowsstate machineorchestrationsaga
Prerequisites: Event-Driven Architectures
Before we start

What you will learn

  • Model a long job as a state machine (steps, states, retries)
  • Compare direct orchestration vs choreography for sagas
  • Handle resume/restart and human approval in workflows
In 60 seconds

An orchestrator (Step Functions, Durable Functions runtime ~ SubOrchestrator, GCP Workflows) tracks a JSON state between steps. Each step invokes a stateless function; the machine remembers where it stopped and can sleep/fail retry/resume — no server holds the state.

Introduction

What is this?

Primitives: states (task, wait, choice, parallel, map, fail, succeed, pass), retry/catch per state, timeout, and the input/output JSON contract between steps. Express vs Standard (AWS: standard exact-once, express high-throughput/at-least-once; Azure Durable: activity/orchestration/entity types; GCP Workflows: steps/join/retry policies). Durable Entities / Durable Objects: stateful actors on serverless (per-entity key concurrency, commands, reminders). Orchestration vs choreography (sagas: orchestrator decides compensation order; choreography: each service compensates its own — fewer points but harder to reason about). Observability/audit: every state transition logged/exported.

Motivation

Why do you need it?

  • State machines turn multi-step jobs into inspectable, resumable graph state.
  • Retry/compensation live in the workflow, not re-implemented per function.
  • Human approval and time-gated events are first-class states.
  • Durable actors compress many tiny stateful services into one consistent system.
Think about it

Mental model

Imagine

A flight itinerary with checkpoints, stopovers and rebooking rules

Think of

Workflow execution state

Each leg (state) is followed by a checkpoint; if the airline (function) fails, the itinerary (workflow) has a rebooking rule and records exactly where you are.

Mechanics

How it works

  • A workflow definition (ASL/JSON/YAML) declares states and their transitions.
  • Each invocation starts a workflow instance with an input payload.
  • A task state calls a function, captures its output, feeds the next state.
  • Failures with retry policy retry; after budget → catch → compensation/fail.
  • Standard execs persist execution history; instantiating a resume reads history.

A checkout-flow state machine.

Validate
Reserve
Charge
Choice
Notify
Hands-on

Examples

ASL sketch of a saga step

ASL sketch of a saga step
"Charge": {
  "Type": "Task",
  "Resource": "arn:aws:states:::lambda:invoke",
  "Parameters": { "FunctionName": "charge", "Payload.$": "$" },
  "Retry": [{ "ErrorEquals": ["TransientError"], "MaxAttempts": 3, "IntervalSeconds": 2 }],
  "Catch": [{ "ErrorEquals": ["States.ALL"], "Next": "Refund" }]
}

One JSON state describes charge + retry + compensation jump — behavior that would otherwise be bespoke code in a service.

Real world

In production

  • ETL orchestration with exponential retry and per-step alerting.
  • Checkout: reserve → charge → ship, compensating refund on failure.
  • Onboarding with a two-week admin-approval wait state.
  • Durable entities for per-user balances with command-based serialization.
Guidelines

Best practices

  • Keep state transitions small and typed; pass only what the next step needs.
  • Use Standard executions where you need audit/retry; Express for fire-and-forget high volume.
  • Define compensation steps explicitly for every unsafe transition.
  • Bound function calls by the workflow's own timeout/lambdas instead of infinite retries.
  • Version workflow definitions; treat schema changes as migration events.
Watch out

Common mistakes

  • Reusing choreography code (compensation lives in consumer) under orchestrator names but without ordering.
  • Storing large payloads in the execution JSON — costs and slows history.
  • Trying to make workflows do fine-grained CPU work; that belongs in functions.
  • Assuming retry is exactly-once; standard is near-solid, express is at-least-once.
When things break

Debugging tips

  • Pull the execution history with state input/output for each transition — the timeline is the debugger.
  • Distinguish retry loops (state x attempts N) from infinite loops (choice misconfigured).
  • Check history size limits; trim payloads your step functions carry.
  • Expose step duration + transition counts as metrics to spot hotspot states.
Interviews

Interview checkpoint

Practice

Hands-on lab

Orchestrate a Resumable Job in a State Machine
40 min · advanced
In progress
Objectives
  • Author a workflow with task + retry + compensation
  • Run it, kill a step, and observe retry/catch semantics
  • Inspect execution history as the ultimate debugger

Orchestrate a Resumable Job in a State Machine

40 min · advanced

0/3 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

Which state primitive adds fan-out?

2

Express/Standard executions differ primarily in:

Go deeper

Official documentation

AWS Step Functions
docs
Azure Durable Functions
docs
Google Workflows
docs