Structured Outputs
Intermediate 13 min read Structured Outputs

Validation & Repair

Typing the model's output like a door check

Parse, validate, and repair on failure: a typed guard rail around generative output. Strict validation with retry-with-errors keeps structured calls honest under noise.

validationparsingretryrepairtype guardzod
Prerequisites: JSON Mode & Enforcement
Before we start

What you will learn

  • Parse + validate model JSON strictly
  • Implement retry-with-error-reply
  • Prevent repair loops from growing unbounded
Introduction

What is this?

Validation is where generative doubt meets typed certainty: parse the model's JSON, validate against the schema (types, enums, constraints), and on failure either repair (send errors back for a corrected generation) or fail honestly. A repair loop with a hard cap keeps latency and tokens bounded while raising successful-parse rates.

Motivation

Why do you need it?

  • Every unvalidated model output is a future prod incident.
  • Repair-with-errors converts most failures into recoveries.
  • Captured failure rates give you a quality metric for prompt/schema changes.
  • Type-safe consumers extend from validated data — no runtime guessing.
Think about it

Mental model

Imagine

Airline check-in with rebooking

Think of

Validation + repair

Boarding pass not matching the seat? The desk re-issues one with the error corrected (retry with feedback) — but only a few times before you miss the aircraft. A capped, message-explicit repair process keeps everyone moving.

Mechanics

How it works

  • Parse the blob; JSON.parse failure → first repair pass.
  • Validate against schema: types, required, enums, bounds.
  • On violation, craft a concise error list honestly.
  • Re-prompt with 'correct these issues: …' and re-validate.
  • Cap retries (1-2); log; fail honestly with the raw blob.

Parse-validate-repair.

Raw output
Parse
Validate
OK?
Repair pass
Failed
Hands-on

Examples

Repair loop sketch (Zod-like)

Repair loop sketch (Zod-like)
const schema = z.object({
  intent: z.enum(["refund", "complaint"]).optional(),
  amount: z.number().min(0).max(100000),
});
 
let raw = await model(prompt);
for (let i = 0; i < 2; i++) {
  const parsed = schema.safeParse(JSON.parse(raw));
  if (parsed.success) return parsed.data;
  raw = await model(
    prompt + "\nYour last reply failed: " + JSON.stringify(parsed.error.issues) +
    "\nReturn only corrected JSON."
  );
}
throw new Error("structured output failed after repair cap");

Two repair passes with concrete errors, then an honest exception.

Real world

In production

  • Every chat-JSON endpoint validates before DB or routing use.
  • Repair rates feed an alert: climbing = schema or prompt decay.
  • Typed DTOs derived from the schema serve UI and integrations.
Guidelines

Best practices

  • Validate the SHAPE, not just 'it's JSON'.
  • Always cap the repair loop; latency is money.
  • Log failure blobs for schema/enum tuning.
  • Alert when typed-parse success drops (schema + model health).
Watch out

Common mistakes

  • Validating with permissive typecasts instead of the schema.
  • Unbounded retry loops that spike cost.
  • Throwing away the error messages when re-prompting (repair is blind).
Root cause analysis

Common issues

Error
Repair orders keep failing
Root cause

Schema + model mismatch

Fix

Re-validate the schema against the true output distribution; relax or re-bin.

Error
Flaky success across seeds
Root cause

Sampling variance vs tight schema

Fix

Lower temperature, tighten schema for the noisy field, keep the retry loop.

When things break

Debugging tips

  • Repeated enum violations → inject the enum list into the prompt.
  • Repair makes it 'worse' → error list confusing; be concise and specific.
  • High failure + low repair → schema unrealistic; check actual model outputs.
Interviews

Interview checkpoint

Practice

Hands-on lab

Repair-Rate Tuning
35 min · intermediate
In progress
Objectives
  • Parse+validate a simulated model output
  • Implement a capped repair loop
  • Measure before/after repair success

Repair-Rate Tuning

35 min · intermediate

0/4 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

Repair loops should be…

2

Feeding validation errors back to the model helps recovery.

Go deeper

Official documentation

Zod
docs