Selenium
Advanced 17 min read Selenium

Flaky Tests & Best Practices

A flaky suite is worse than no suite — it teaches everyone to ignore red builds. Diagnose the common flakiness drivers (waits, shared state, environment drift, order dependence) and apply the Page Object Model, deterministic waits and CI-safe headless execution.

flaky testsPage Object Modelheadlessretrydeterminismtest isolation
Prerequisites: Implicit vs Explicit Waits Selenium Grid & Parallel Testing
Introduction

What is this?

A flaky test passes locally and fails on CI, or fails one run and passes the next, with no code change. The usual drivers are poor synchronization, shared mutable state, environment drift, test-order dependence and fragile locators. Best practice is to make every test deterministic and isolated: explicit waits on real application state, no inter-test dependencies, environment-agnostic configuration, headless CI runs, and a Page Object Model that keeps selectors and flows in one place.

Motivation

Why do you need it?

  • Flaky tests destroy trust in the pipeline — people start ignoring failures and real regressions slip through.
  • Deterministic tests fail fast with an exact reason instead of randomly wasting developer hours.
  • Isolation lets you re-run a single test or run tests in parallel without interference.
  • The Page Object Model centralizes locators so a UI change breaks one file instead of hundreds of tests.
  • Fast, headless, parallel runs give fast feedback and are cheap enough to run on every commit.
Think about it

Mental model

Imagine

A flaky smoke detector

Think of

An unreliable test suite

If the alarm goes off randomly, people disable it — the one time it matters, it is silent. A flaky test suite gets 'disabled' the same way, and real regressions ship.

Mechanics

How it works

  • Wait on application state with explicit waits instead of fixed sleeps so timing never gates correctness.
  • Keep tests independent: create their own data and never rely on a previous test's login or cart.
  • Encapsulate pages in Page Objects: locators and actions live in a class, tests only describe intent.
  • Run headless in CI with a pinned browser/driver version and a dedicated clean environment.
  • Use retry at the runner level only as a safety net while fixing the underlying nondeterminism.

Anatomy of a stable test vs a flaky test.

Deterministic setup
Explicit wait
Page Object
Assert outcome
Result
Failure evidence
Trusted signal
Reference

Key commands

python -m pytest tests/ --flaky-rerun 1

Retry failed tests once as a safety net (pytest-rerunfailures) while you fix the root cause.

mvn test -Dheadless=true

Run the suite headless; the driver factory reads this flag from system properties.

python -m pytest tests/test_login.py -x

Stop at the first failure to see whether order or state causes cascading errors.

python -m pytest --html=report.html --self-contained-html

Generate a self-contained HTML report with failure evidence (pytest-html).

ps aux | grep chromedriver

Confirm no leaked driver processes remain after a run.

Hands-on

Examples

A Page Object test with headless Chrome

A Page Object test with headless Chrome
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
 
 
class LoginPage:
    def __init__(self, driver):
        self.driver = driver
        self.user = (By.ID, "email")
        self.password = (By.ID, "password")
        self.submit = (By.CSS_SELECTOR, "button[type='submit']")
        self.message = (By.ID, "message")
 
    def open(self, base_url):
        self.driver.get(base_url + "/login")
        return self
 
    def login(self, user, password):
        self.driver.find_element(*self.user).send_keys(user)
        self.driver.find_element(*self.password).send_keys(password)
        self.driver.find_element(*self.submit).click()
        return self
 
    def message_text(self):
        return self.driver.find_element(*self.message).text
 
 
def build_driver(headless=True):
    options = Options()
    if headless:
        options.add_argument("--headless=new")
    return webdriver.Chrome(options=options)
 
 
def test_valid_login():
    driver = build_driver()
    try:
        page = LoginPage(driver).open("https://example.com").login("alice", "s3cret")
        assert page.message_text() == "Welcome back"
    finally:
        driver.quit()

LoginPage wraps the locators and the flow in one class, so the test only expresses intent: open, log in, assert. build_driver centralizes browser options (headless in CI). The try/finally guarantees quit() even when the assertion fails, and the single assertion keeps the failure reason precise.

Real world

In production

  • Run the smoke set headless on every pull request for fast feedback; run the full regression nightly.
  • Store screenshots and DOM snapshots on failure so developers debug from artifacts, not guesses.
  • Pin Chrome and ChromeDriver versions in CI to eliminate 'works on my machine' drift.
  • Maintain a quarantine folder for unstable tests while they are diagnosed and fixed.
Guidelines

Best practices

  • Keep one assertion per logical check; long tests obscure what actually broke.
  • Keep tests browser-agnostic through a driver factory that reads configuration.
  • Run tests in random order locally to expose hidden inter-test dependencies.
  • Pin versions (browser, driver, Selenium) everywhere — dev, CI and grid.
  • Prefer stable test ids (data-testid) over presentational classes for locators.
Watch out

Common mistakes

  • Fixing flakiness with longer sleeps, which slows the suite and still races on loaded CI machines.
  • Letting tests share account state, so ordering and parallel runs produce false failures.
  • Asserting on full page text or brittle CSS chains that change with every deploy.
  • Ignoring flakes 'for now' — the suite quietly rots until no one trusts it.
Root cause analysis

Common issues

Error
Element ... is not clickable at point (x, y)
Root cause

The element is covered by a banner, toast or sticky header at click time.

Fix

Wait for the overlay to disappear, scroll the element into view, or use the Page Object's force-click helper.

Error
Session timed out or not created
Root cause

The CI machine is overloaded or the browser/driver crashed under parallel load.

Fix

Pin versions, reduce parallelism, add memory, and ensure every driver is quit.

Error
Test passes locally, fails only on CI
Root cause

Environment drift — different browser version, network speed or screen size.

Fix

Match CI to local (pinned image, headless, fixed viewport) and replace timing assumptions with explicit waits.

When things break

Debugging tips

  • Re-run the failing test alone (pytest -k or -Dtest); if it passes in isolation, look for shared state.
  • Compare the passing and failing DOM by dumping page_source at the failure point in both runs.
  • Grep CI logs for the first failure, not the loudest — cascade failures hide the root cause.
  • Watch a recorded run (video or Selenium Replay) to see the sequence leading to the flake.
Interviews

Interview checkpoint

Practice

Hands-on lab

De-Flake a Real Suite
35 min · advanced
In progress
Objectives
  • Reproduce a deliberately flaky test
  • Replace sleep-based timing with explicit waits
  • Refactor toward the Page Object Model
  • Run the stabilized test repeatedly to prove determinism

De-Flake a Real Suite

35 min · advanced

0/6 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/3
1

Which is the strongest fix for a test that races a slow network?

2

A test that passes alone but fails when run after other tests indicates an isolation problem.

3

After enabling headless CI runs, several tests fail with size/visibility errors that never fail locally. What is the best fix?

Quick reference

Cheat sheet

Stability rules
Explicit waitsWait for state, never sleep
No shared stateOwn data per test
Pin versionsBrowser, driver, Selenium
Headless in CIFixed viewport, no display needed
Capture artifactsScreenshot + DOM on failure
Random orderExpose hidden dependencies
Flake triage
pytest -k test_loginIsolate one test
Re-run in isolationPassing alone = shared state
dump page_sourceCompare passing vs failing DOM
driver.save_screenshotVisual evidence at failure
--flaky-rerun 1Safety net only, fix root cause
Quarantine tagPark unstable tests while fixing
Go deeper

Official documentation

Page Object Models
docs
Discouraged practices
docs
Encouraged practices
docs
W3C WebDriver Specification
spec