Guardrails
Advanced 13 min read Guardrails

PII Detection & Handling

Privacy you can prove

Personally identifiable information needs detection, redaction, and data-governance rules: mask on input, strip on output, and keep PII out of model context and training. Detection quality decides whether privacy is real or theater.

piiredactionmaskingprivacygdpranonymizationtokenization
Prerequisites: Output Filtering
Before we start

What you will learn

  • Detect PII classes reliably
  • Mask on input, strip on output
  • Govern PII in context and logs
Introduction

What is this?

PII handling means recognizing identifiable data (names, emails, addresses, IDs, phone) with detectors, redacting/masking it before it reaches context or logs, and stripping it from responses. Detection quality is the crux: regex + entity models + feedback loops, measured against labeled test sets.

Motivation

Why do you need it?

  • Leakage to logs or training is a compliance and trust failure.
  • Masking keeps context useful while removing identity.
  • Stripped output prevents accidental disclosure to users.
  • Measured detection keeps the practice honest.
Think about it

Mental model

Imagine

A hospital's patient chart system

Think of

PII handling

Names appear masked ('Patient #7') in shared rounds; charts travel only with authorization; disposal shreds identity. The system still works clinically (masked context) without leaking who is who. Your LLM pipeline needs those same walls.

Mechanics

How it works

  • Detect: regex lists + NER model for PII classes.
  • Mask on input: replace tokens with placeholders (preserving structure).
  • Context rule: allow masked form only in prompts.
  • Strip on output: scan final text; redact or refuse.
  • Govern logs: no raw PII; retention and access rules.

PII lifecycle.

Detect
Mask
Context
Strip
Audit
Real world

In production

  • Customer support routes mask names/emails before model calls.
  • Healthcare/finance pipelines redact on chat and export.
  • Eval datasets scrub PII before they ship.
Guidelines

Best practices

  • Detect with layered tools (regex + NER) and label coverage.
  • Mask preserving length/structure where the model needs it.
  • Strip output regardless of whether input was masked.
  • Test detection against labeled PII sets (recall-first).
Watch out

Common mistakes

  • Regex-only detection with 60% recall and quiet confidence.
  • Masking input but leaking restored values in the prompt.
  • PII parked in logs 'for now'.
Root cause analysis

Common issues

Error
Model needs unmasked context
Root cause

Structure lost in masking

Fix

Mask to controlled placeholders preserving layout.

Error
PII in eval/training data
Root cause

Dataset not scrubbed

Fix

Run scrubbing pipeline + regression cases.

When things break

Debugging tips

  • Missed detection → label set inspection; add entity patterns.
  • Over-masking breaks the task → context-aware rules.
  • PII reappears in output → restoration step leaking; strip hard.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

The strongest PII posture is…

2

Regex alone reliably covers all PII classes.

Go deeper

Official documentation

PII detection (Presidio)
docs