Automation Testing
Beginner 14 min read Automation Testing

Test Frameworks

A test framework supplies the runner, assertion library, lifecycle hooks and fixtures that turn raw test functions into a maintainable suite. Learn how Jest, pytest, JUnit and TestNG structure tests and what makes them production-grade.

jestpytestjunittestngtest runnerfixturesassertionstest lifecycle
Prerequisites: Automation Fundamentals
Introduction

What is this?

A test framework standardizes how tests are written, discovered, executed and reported. It provides the runner (CLI, watch mode, parallel execution), an assertion API, setup/teardown lifecycle hooks, and fixtures for shared state. Jest dominates the JavaScript world, pytest is the Python standard, and JUnit/TestNG are the JVM workhorses - all solve the same problems with slightly different idioms.

Motivation

Why do you need it?

  • Runners handle discovery, ordering, parallel execution and exit codes so CI can trust the result.
  • Lifecycle hooks (beforeEach, setUp, fixtures) eliminate duplicated setup and keep tests isolated.
  • Rich assertion APIs produce readable actual-vs-expected failures instead of raw exceptions.
  • Plugins and reporters connect suites to coverage, snapshots, sharding and CI dashboards.
  • The community around a mature framework ships mocks, parameterization and browser integrations you would otherwise build yourself.
Think about it

Mental model

Imagine

A restaurant kitchen

Think of

A test framework

The kitchen standardizes prep: the runner is the line-up of stations, beforeAll/beforeEach are the mise en place, the assertion library is the plating checklist, and the reporter is the ticket that tells the fate of every dish. Ingredients are never prepared twice, and every station checks the plate before it leaves.

Mechanics

How it works

  • You write test functions and register them with annotations or naming conventions the runner recognizes.
  • The runner scans configured paths (testMatch, rootdir, conftest.py), collecting tests before running any.
  • Hooks run in order per suite and per test: setup, test body, assertion, teardown - even on failure.
  • Each test executes in an isolated scope; assertions that throw mark the test failed and the runner continues or stops per policy.
  • The reporter aggregates results into stdout, JUnit XML and exit codes that the pipeline can gate on.

The lifecycle of a single test inside a framework.

Collect tests
Setup hooks
Execute test body
Assertion check
Teardown
Report and exit code
Reference

Key commands

npx jest --listTests

Print every test file the runner would execute, useful for debugging discovery.

src/validateEmail.test.ts
src/cart.test.ts
pytest --fixtures

List all fixtures available in the session with their docstrings.

pytest -q --junitxml=results.xml

Run quietly and emit JUnit XML that CI tools and dashboards can parse.

4 passed in 0.32s
npx jest --runInBand

Run tests sequentially in one process to rule out parallel-state bugs.

mvn test -Dtest=CartTest#testAddItem

Run a single JUnit test method with Maven's test filter.

pytest --maxfail=2

Stop the run after two failures so flaky or order issues surface quickly.

Hands-on

Examples

pytest fixtures, parametrize and hooks

pytest fixtures, parametrize and hooks
import pytest
 
@pytest.fixture
def cart():
    """A fresh cart for every test - no shared state."""
    return {"items": [], "total": 0}
 
@pytest.mark.parametrize("qty,expected", [(0, 0.0), (2, 19.98), (5, 49.95)])
def test_add_item(cart, qty, expected):
    cart["items"].extend(["coffee"] * qty)
    cart["total"] = round(qty * 9.99, 2)
    assert cart["total"] == expected
 
def test_empty_cart_total(cart):
    assert cart["total"] == 0

The cart fixture is invoked by pytest for every test that asks for it, giving a clean isolated object. @pytest.mark.parametrize runs test_add_item three times with different (qty, expected) tuples, so the same logic is verified across inputs without duplicating code. pytest discovers the file because it is named test_*.py.

Real world

In production

  • Frameworks generate JUnit XML consumed by Jenkins, GitHub Actions and reporting tools like Allure.
  • pytest fixtures and JUnit setup methods provision databases and API stubs and tear them down per test.
  • Parameterized suites run the same test across configurations (browser, locale, feature flag) from one code path.
  • Snapshot testing in Jest locks rendered output to catch unintended UI and markup changes in review.
Guidelines

Best practices

  • One test per behavior; name it as a sentence so the report reads like a spec.
  • Use fixtures and setup hooks instead of test inheritance or module-level mutable globals.
  • Parameterize data variations but keep the number of cases high-signal, not exhaustive.
  • Pin dependency versions and run on a fixed Node/Python runtime in CI for reproducible results.
  • Keep assertion messages meaningful so a failing test explains the contract, not just a mismatch.
Watch out

Common mistakes

  • Sharing mutable state through module-level globals, making tests order-dependent.
  • Testing implementation details (private methods, internal functions) so refactors break valid tests.
  • Asserting too little, so the test passes even when behavior regresses.
  • Copying setup into every test instead of using hooks, producing a suite that is hard to maintain.
Root cause analysis

Common issues

Error
ModuleNotFoundError: No module named 'pytest'
Root cause

pytest is not installed in the active environment.

Fix

Install with python -m pip install pytest, then re-run.

Error
fixture 'cart' not found
Root cause

The fixture is defined in another module and not imported, or the name is misspelled.

Fix

Move the fixture to conftest.py or import it where the test lives.

Error
Error: Test environment must contain a valid Node version
Root cause

The runner executed under an old or missing Node runtime.

Fix

Use the engine the suite targets, e.g. nvm use 20 or a fixed node:20 CI image.

When things break

Debugging tips

  • Run pytest --fixtures and npx jest --listTests to confirm exactly what is being executed.
  • Isolate order effects with --runInBand (Jest) or --dist no (pytest) to reproduce locally.
  • Use -vv with --tb=short to see the failing assertion diff plus a tight traceback.
  • Temporarily skip a failing test with @pytest.mark.skip to confirm the rest of the suite is green.
Interviews

Interview checkpoint

Practice

Hands-on lab

Parametrized Cart Tests with Fixtures
30 min · beginner
In progress
Objectives
  • Install pytest in a virtual environment
  • Define a reusable fixture for isolated test state
  • Parametrize a test across multiple input sets
  • Run the suite and emit JUnit XML

Parametrized Cart Tests with Fixtures

30 min · beginner

0/5 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/3
1

Which hook runs before EVERY test in a suite?

2

A test that depends on another test's side effects is considered well-isolated.

3

Which pytest marker runs the same test function with multiple sets of inputs?

Quick reference

Cheat sheet

Framework idioms side by side
describe / test / expectJest basic structure
def test_x() / assertpytest structure
@Test / @BeforeEachJUnit 5 annotations
@Test / @BeforeMethodTestNG annotations
beforeEach / afterEachPer-test hooks (Jest)
@pytest.fixture / conftest.pypytest shared fixtures
Go deeper

Official documentation

Jest Documentation
docs
pytest Documentation
docs
JUnit 5 User Guide
docs
TestNG Documentation
docs