Automation Testing
Advanced 15 min read Automation Testing

Parallel Execution & Sharding

A suite that takes 40 minutes is a feedback loop nobody uses; a sharded suite takes minutes. Learn how workers, shards and per-worker isolation turn a single slow run into many fast parallel runs - and the isolation rules that keep them deterministic.

parallel testingshardingpytest-xdistjest workersplaywright workerstest isolationCI matrix
Prerequisites: Test Frameworks CI/CD Test Integration
Introduction

What is this?

Parallel execution runs tests on multiple workers, threads, processes or machines at once. Sharding splits a suite across N runners, each executing a disjoint subset, and merges their reports afterward. Frameworks like pytest-xdist, Jest workers, TestNG and Playwright workers provide this natively, but parallelism only pays off when tests are truly isolated - no shared state, no shared files, no order dependencies.

Motivation

Why do you need it?

  • Wall-clock time drops roughly linearly with workers, turning a 40-minute suite into a 5-minute gate.
  • Parallel workers keep a feedback loop fast enough that developers run tests on every change.
  • Sharding across CI machines lets browser-heavy E2E suites run on as many runners as your budget allows.
  • Isolation enforced for parallelism also removes order-dependent flakes from serial runs.
  • Independent cases are retryable per worker, so a single flaky row does not poison the whole run.
Think about it

Mental model

Imagine

A toll plaza

Think of

Parallel workers

A single toll booth (one worker) processes cars one at a time; 12 booths (12 workers) process 12 cars at once - as long as every car has its own lane (isolation) and cannot block the others. If cars were forced to share one lane, the plaza would just be chaos.

Mechanics

How it works

  • The runner divides the suite into subsets: pytest-xdist splits by test node, Jest assigns tests to workers, CI shards split by list.
  • Each worker process spins up its own runtime, so state (imports, DB connections, page instances) is not shared.
  • Tests run concurrently; the framework balances load and, for browser tests, uses one browser context per worker.
  • Each worker produces its own results (JUnit XML fragment, log, screenshots) keyed by worker ID.
  • Reports are merged into one dataset; CI shards are reassembled by the pipeline so the status check is one signal.

One suite split across workers and reassembled into one report.

Full suite
Split into subsets
Worker processes
Run concurrently
Per-worker results
Merge and gate
Reference

Key commands

npx playwright test --shard=2/4

Run only the second quarter of the suite (for CI shard jobs).

pytest -n 8

Run with pytest-xdist across 8 parallel worker processes.

8 workers [200 items]
200 passed in 12.34s
npx jest --maxWorkers=50%

Cap Jest workers at half the CPUs to leave headroom in shared CI.

npx jest --listTests > shard-1.txt

Produce the test list that a shard-splitting script will divide.

pytest --dist loadgroup

Keep tests with the same marker or group on the same worker to avoid resource contention.

npx playwright test --workers=1

Force serial execution to debug an ordering or isolation bug.

Hands-on

Examples

Playwright sharding across GitHub Actions runners

Playwright sharding across GitHub Actions runners
name: e2e-sharded
on:
  pull_request:
 
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npx playwright test --shard=${{ matrix.shardIndex }}/4
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: test-report-${{ matrix.shardIndex }}
          path: |
            test-results/
            playwright-report/

The matrix creates four parallel jobs, each running a quarter of the suite via --shard=index/4. fail-fast: false ensures one shard's failure does not cancel the others. Each shard uploads its own traces and report, so a failure points at the exact shard and case. Combined, the four jobs finish in roughly a quarter of the serial time.

Real world

In production

  • Big E2E suites shard across 4-8 CI runners so PR feedback stays under five minutes.
  • pytest-xdist load-scopes CPU-heavy unit suites onto all available cores locally and in CI.
  • Per-worker browser contexts and isolated databases keep parallel UI suites deterministic.
  • Fail-fast off plus per-shard artifacts means one flaky worker can be retried without rerunning everything.
Guidelines

Best practices

  • Enforce isolation first: no shared mutable globals, no fixed ports, no shared files or DB records.
  • Make tests idempotent and self-seeding - each test creates and cleans up its own data.
  • Use fail-fast: false in CI matrices so one shard's failure does not waste the other shards' results.
  • Tag resource-hungry tests (for example performance) so the scheduler groups them onto dedicated workers.
  • Measure the split: keep shards balanced by runtime, not just by count, using recent timing data.
Watch out

Common mistakes

  • Splitting by count when runtime varies wildly, so one shard finishes in 2 minutes and another in 20.
  • Tests sharing fixed ports or fixture files, which collide the moment two workers run concurrently.
  • Using a single shared database without per-test isolation, causing phantom failures only in parallel.
  • Setting fail-fast: true on a matrix, so one flaky shard cancels the remaining work.
Root cause analysis

Common issues

Error
Error: listen EADDRINUSE: address already in use :8080
Root cause

Two workers bind the same port on the same machine.

Fix

Assign each worker a unique port (env var per worker/shard) or let the app pick an ephemeral port.

Error
pytest: error: unrecognized arguments: -n 8
Root cause

pytest-xdist is not installed.

Fix

Install with pip install pytest-xdist, then re-run pytest -n 8.

Error
worker 'gw0' crashed
Root cause

A worker process crashed (segfault, OOM) rather than failing a test.

Fix

Reproduce serially on the crashing test, reduce per-worker load (memory/threads), or add a memory cap.

When things break

Debugging tips

  • Force serial with --workers=1 or -n 1: if the suite passes serially but fails parallel, it is an isolation bug.
  • Key artifacts by worker or shard (shardIndex, worker ID) so failures map to a specific worker.
  • Run a smaller slice (--shard=1/20) to reproduce a shard-only failure on your laptop.
  • Add the worker ID to log lines so interleaved output can be attributed to the right process.
Interviews

Interview checkpoint

Practice

Hands-on lab

Shard a Playwright Suite Across Four Runners
45 min · advanced
In progress
Objectives
  • Generate or use a multi-file Playwright suite
  • Add a 4-way shard matrix to the CI workflow
  • Upload per-shard artifacts keyed by index
  • Verify total wall time drops and reports merge

Shard a Playwright Suite Across Four Runners

45 min · advanced

0/5 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/3
1

Which condition is REQUIRED before a suite can run reliably in parallel?

2

fail-fast: true is a good default for a multi-shard CI matrix.

3

pytest -n 8 fails with 'unrecognized arguments: -n 8'. What is the fix?

Quick reference

Cheat sheet

Parallel execution quick reference
pytest -n 88 pytest-xdist workers
pytest --dist loadgroupGroup tests to workers
npx jest --maxWorkers=50%Cap Jest parallelism
npx playwright test --shard=2/4Run one CI shard
--workers=1Serial fallback for debug
fail-fast: falseKeep all shards running
Go deeper

Official documentation

pytest-xdist - Process Parallelism
docs
Playwright - Parallelism & sharding
docs
Jest - CLI maxWorkers
docs
TestNG - Parallel tests
docs