Data Handling
Intermediate 13 min read Data Handling

Handling JSON at Scale

Parsing what everyone hands you

JSON is the wire format of model APIs, tool calls and structured outputs. Parsing large streams, validating against schemas, and normalizing messy payloads cleanly separates robust AI services from brittle ones.

JSONparsingschemastreamingndjsonvalidationnormalization
Prerequisites: Document Processing
Before we start

What you will learn

  • Parse large and streaming JSON safely
  • Validate payloads against schemas
  • Normalize messy provider outputs into typed records
Introduction

What is this?

JSON handling for AI means three battle-tested tasks: parsing documents and streams without blowing memory, validating payloads against a schema so shape drift is caught, and normalizing messy provider/model output into typed records your pipeline can trust. Nearly every failure in structured-output pipelines is a JSON problem at heart.

Motivation

Why do you need it?

  • Model tool-call output arrives as JSON text; one unescaped string or truncated stream breaks the whole path.
  • Provider responses evolve; schema validation turns silent shape drift into a loud, catchable error.
  • Streaming NDJSON lets you process huge exports a record at a time instead of holding them all.
  • Normalization at the edge keeps the rest of your code simple and typed.
Think about it

Mental model

Imagine

A customs declaration form

Think of

JSON validation

The inspector checks your form before anything enters the country: every field present, right format, nothing smuggled. A JSON validator plays customs agent between untrusted output and your business logic.

Mechanics

How it works

  • json.loads parses bytes/strings into native structures; bad syntax raises loudly with a line/col hint.
  • For huge files, stream with `json.load` over a file or iterate NDJSON lines, one object at a time.
  • JSON Schema (or pydantic) declares required fields, types and constraints; validation tests payloads against it.
  • normalization converts variant shapes (nulls, missing keys, type changes) into a single contract.
  • Escaping/truncation bugs in model output are handled at parse+repair time with tolerant decoders where needed.

The JSON pipeline.

Raw payload
Parse
Validate
Normalize
Typed records
Hands-on

Examples

Strict validation with pydantic

Strict validation with pydantic
from pydantic import BaseModel, Field
 
class ToolResult(BaseModel):
    ok: bool
    value: str = Field(min_length=1)
    meta: dict = Field(default_factory=dict)
 
# Invalid — missing 'ok' raises ValidationError instead of breaking later
ToolResult.model_validate({"value": "x"})

pydantic both validates and normalizes — wrong types become loud Errors at the boundary, and output becomes typed objects your pipeline can use without paranoia.

Real world

In production

  • A function-calling loop validates each tool JSON before dispatch; malformed output triggers a corrective retry rather than a crash.
  • A data-ingest feeds a million-line NDJSON corpus through a streaming parser under a memory ceiling.
  • A structured-output service validates model output against the declared JSON Schema and re-asks on failure.
Guidelines

Best practices

  • Validate at the boundary; never trust provider or model output to be well-formed.
  • Stream records for large datasets instead of loading whole files.
  • Use pydantic/JSON Schema so type errors are explicit, not implicit.
  • Log parsing errors with the raw snippet (redacted) for fast triage.
Watch out

Common mistakes

  • json.loads on a 5GB file and OOM at 3am.
  • Indexing dict fields by name without validating structure — one key rename breaks silently.
  • Silently swallowing parse errors and shipping empty results that look correct.
  • Treating all model tool-call JSON as well-formed when a truncated stream says otherwise.
Root cause analysis

Common issues

Error
json.JSONDecodeError with model output
Root cause

Truncated completion or unescaped control characters in text

Fix

Repair with a tolerant parser or reask with structured output mode; cap output length.

Error
ValidationError chain across providers
Root cause

Provider-specific field naming or added fields

Fix

Normalize per-provider in the adapter and validate the common contract after.

Error
Memory blows up parsing a big corpus
Root cause

Whole file loaded instead of streamed

Fix

Use incremental parsing / NDJSON iteration with a bounded record size.

When things break

Debugging tips

  • Print the raw JSON once — most 'why won't it parse' answer themselves.
  • Track an error with the character offset json.loads reports; look just before it.
  • If NDJSON half-imports, re-run after fixing one malformed line and log its index.
  • For model output, diff raw vs parsed to catch unescaped quotes from truncation.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

What happens when a payload fails JSON Schema validation?

2

Loading an entire large JSON file into memory is the best way to process it.

Go deeper

Official documentation

JSON — MDN
docs
pydantic docs
docs