Guardrails
Intermediate 12 min read Guardrails

Output Filtering

The last gate before delivery

Output filters police what leaves the system: toxicity, unsafe content, leaked private data, and rule-breaking claims. They run after generation and before delivery — the last line of policy defense.

output filtermoderationtoxicitydata leakagepost-generationblocklist
Prerequisites: Guardrails Overview
Before we start

What you will learn

  • Filter toxic and unsafe output
  • Detect leaked private data in output
  • Handle filtered outputs gracefully
Introduction

What is this?

Output filtering runs after generation: moderation classifiers for toxicity, scanners for PII and secrets, schema/format validators, and even factuality spot-checks. Filtered content is either reparsed, regenerated, or replaced with a safe fallback rather than delivered raw.

Motivation

Why do you need it?

  • Generation can fail even under good prompts.
  • Leaked data leaves the system only at delivery — catch it before.
  • Output checks are testable policy, independent of the model.
  • Regulatory requirements often call for them explicitly.
Think about it

Mental model

Imagine

Loading inspection before shipping

Think of

Output filter

Every parcel leaves the warehouse through the scanner: labeled correctly (format), contains no prohibited goods (content), addresses right (PII). A bad parcel gets re-routed or rejected here — nothing ships off-policy.

Mechanics

How it works

  • Run moderation classifier on generated text.
  • Scan for PII/secrets/patterns on the way out.
  • Validate format/schema compliance.
  • On failure: sanitize, regenerate once, or fallback.
  • Log the decision and outcome.

Post-generation gate.

Output
Moderate
Scan
Pass?
Fallback
Real world

In production

  • Public chat filters profanity and harassment on output.
  • Enterprise apps strip PII before export.
  • Finance tools reject unsupported compliance claims.
Guidelines

Best practices

  • Classify the failure: moderate, sanitize, regenerate, or refuse.
  • Design fallbacks so the filter doesn't 'hang' the user.
  • Measure false positive + miss rates; tune thresholds.
  • Log filter actions for abuse analysis.
Watch out

Common mistakes

  • Delivering flagged content anyway ('it only fired sometimes').
  • No fallback — legit queries get blank walls.
  • Filtering only the happy path, missing streaming overs.
Root cause analysis

Common issues

Error
Filtered output blocks legit use
Root cause

Threshold too tight

Fix

Tune on traffic; add severity-based handling.

Error
Regeneration loops on filtered output
Root cause

No fallback logic

Fix

Regen once, then fixed safe reply.

When things break

Debugging tips

  • Users see barren replies → false-positive falls in the classifier; tune + fallback.
  • PII slips once → scanner gap; add rule + regression test.
  • Stream shows leaks before filter → check streaming path filters too.
Interviews

Interview checkpoint

Practice

Hands-on lab

Redact Sensitive Output
30 min · intermediate
In progress
Objectives
  • Detect emails and long digit runs in output
  • Replace matches with a redaction token
  • Verify no raw PII leaks after redaction

Redact Sensitive Output

30 min · intermediate

0/3 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

Output filtering happens…

2

A filter should always deliver legally-flagged content.

Go deeper

Official documentation

Moderation API (OpenAI)
docs