Manual Testing
Intermediate 12 min read Manual Testing

Test Reporting & Metrics

Metrics turn executed tests into a decision-ready story. Learn which numbers actually matter (pass rate, coverage, defect density, escape rate), how to avoid vanity metrics, and how to present results so the release owner can say go or no-go with confidence.

metricsreportingpass ratedefect densityescape ratecoverage
Prerequisites: Test Execution & Tracking Test Planning & Strategy
Introduction

What is this?

Test reporting summarizes what was tested, what passed, what failed and what risk remains, in numbers and trends a decision-maker can act on. The useful metrics are pass rate, execution progress, coverage, defect density, defect aging and escape rate. A good report explains the why behind the numbers and states the remaining risk, not just a table of counts.

Motivation

Why do you need it?

  • Trends beat snapshots: a single pass rate hides whether quality is improving or sliding.
  • Decision-makers need risk stated plainly, not buried in a spreadsheet.
  • Honest metrics build trust; cherry-picked vanity metrics destroy it.
  • Escape rate exposes gaps in testing that raw pass rate hides.
  • Standardized reports let a new stakeholder understand any release in five minutes.
Think about it

Mental model

Imagine

A car's dashboard

Think of

A test report

You would not drive based only on the speedometer. The dashboard shows fuel, temperature, warnings and distance — together they tell you whether the trip is safe. A test report is a dashboard: pass rate, coverage, open defects, aging and risk, read together.

Mechanics

How it works

  • Collect execution outcomes, defect records and coverage data from the run.
  • Compute the core metrics: pass rate, execution progress, coverage, defect density and aging.
  • Compare against the previous release or the same period last sprint to show the trend.
  • Write the story: what was tested, what failed, what risk remains, and a recommendation.
  • Present to stakeholders before the release decision and archive for audit history.

From raw execution data to a release decision.

Execution data
Compute metrics
Show trends
Write the story
Recommendation
Release decision
Reference

Key commands

grep -c PASS run-log.txt && grep -c FAIL run-log.txt

Pull the raw counts for pass rate.

gh issue list --label bug --state open

Get the open defect count for the report.

DF-102  Checkout tax off by 0.01  bug
python3 -c "print(f'pass rate: {42/(42+3)*100:.1f}%')"

Compute pass rate from counts.

pass rate: 93.3%
gh issue list --label bug --search 'is:open sort:created-asc'

Age the backlog to show how long defects have been open.

curl -sf https://staging.example.com/api/health | jq '.status'

Confirm environment health to explain any blocked-test counts.

Hands-on

Examples

Computing a release report from a run log

Computing a release report from a run log
# Count outcomes from the run log
passes=$(grep -c PASS run-log.txt)
fails=$(grep -c FAIL run-log.txt)
total=$((passes + fails))
pass_rate=$(awk "BEGIN{printf \"%.0f\", $passes*100/$total}")
 
# Defect metrics
open=$(gh issue list --label bug --state open | wc -l)
escaped=$(gh issue list --label 'escape' --state closed | wc -l)
 
echo "Pass rate   : $pass_rate%"
echo "Open defects: $open"
echo "Escapes     : $escaped"
echo "Risk        : tax rounding (DF-102) blocks full sign-off"

The report keeps it to five lines a decision-maker can read in one breath: pass rate, open defects, escapes, and the one risk that matters. The numbers come from the same logs the team already maintains.

Real world

In production

  • Release dashboards show pass rate, open P0/P1 and escape rate trends across releases.
  • Defect density (defects per module or per 1k LOC) flags which modules are chronic problems.
  • Escape rate reviews after each release calibrate where testing effort should shift next cycle.
  • Aging charts surface long-open defects that product and engineering are silently deferring.
Guidelines

Best practices

  • Report the trend, not just today's number — one release tells you little.
  • Define every metric once and compute it the same way every time.
  • Separate product defects from environment blocks so the numbers stay honest.
  • State remaining risk explicitly and name the owner of each open item.
  • Keep the report short enough to read in five minutes and archive it for audits.
Watch out

Common mistakes

  • Reporting 100% pass rate while blocking half the cases as 'blocked'.
  • Using coverage % as a quality metric without considering whether tests are meaningful.
  • Cherry-picking the one metric that looks good and hiding the rest.
  • Writing a report with numbers but no interpretation or recommendation.
Root cause analysis

Common issues

Error
Stakeholders only look at pass rate
Root cause

Report leads with one headline number

Fix

Lead with a risk paragraph and pair pass rate with open P0/P1 and escape trend.

Error
Pass rate says 100% but production incidents occur
Root cause

Tests miss real-world paths or data

Fix

Shift effort toward scenario and exploratory coverage of high-risk flows.

Error
Metrics change meaning between releases
Root cause

Definitions not fixed

Fix

Document each metric and its formula in a shared glossary.

When things break

Debugging tips

  • If pass rate looks too good, audit for skipped or unrecorded cases.
  • When defect count spikes, break it down by module and by phase to find where quality is slipping.
  • If escape rate is high, the test design, not the execution, is the gap — revisit coverage of changed code.
  • Compare open-defect aging against the release date to expose deferred risk.
Interviews

Interview checkpoint

Practice

Hands-on lab

Produce a Release Readiness Report
35 min · intermediate
In progress
Objectives
  • Compute pass rate and defect metrics from a run log
  • Show the trend against the previous release
  • Write a five-line summary with remaining risk
  • Present a go/no-go recommendation

Produce a Release Readiness Report

35 min · intermediate

0/5 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/3
1

Which metric best reveals gaps that pass rate hides?

2

Reporting pass rate without blocked cases is a fair picture.

3

Two releases show pass rate 92% and 91%, but the second release shipped a critical defect that escaped. What is the better conclusion?

Quick reference

Cheat sheet

Core test metrics
Pass ratePassed / executed, trend over time
Execution progressExecuted vs planned cases
Defect densityDefects per module / 1k LOC
Defect agingDays open, oldest first
Escape rateProduction defects vs total
Go deeper

Official documentation

ISTQB — Test Metrics and Reporting
docs
DORA — Four Key Metrics
docs
Atlassian — Agile Testing Metrics
docs