Debugging API Errors
Read the wire before rebuilding the world
Most model-integration failures are diagnosable from the HTTP exchange: status code, response body, headers and timing. This is a methodical playbook for reading those signals.
What you will learn
- Diagnose failures from status code plus response body
- Use request IDs and timing to localize the fault
- Reproduce API issues with minimal curl probes
What is this?
Debugging an AI API integration is reading four signals in order: the status code (what class of failure), the response body (what's wrong specifically), the headers (rate limits, request IDs, retry hints), and the timing (where latency is lost). Most 'the model is broken' reports are actually config, auth, proxy or retry bugs visible at the wire level.
Why do you need it?
- Status + body identifies 80% of issues before any code is touched.
- Request IDs let you correlate a failing call across provider, gateway and your logs.
- Timing breakdown (connect vs TTFT vs completion) shows whether the problem is your network, the proxy or the model.
- Curl reproductions strip away SDK and framework noise to expose the real exchange.
Mental model
A doctor's vital signs
Status, body, headers, timing
You check vitals first — temperature, pulse, color — before guessing at diseases. API debugging is the same: quick observable signals before deep investigation.
How it works
- Category the code: 4xx = your request is wrong; 429 = pressure; 5xx = their side; timeout = handled on your side.
- Read the body; providers write field-level validation errors you can act on directly.
- Grab request IDs from headers; they name the exact transaction across systems.
- Time the phases: DNS+TCP+TLS, first byte, then completion.
- Reduce: strip streaming, reduce a huge prompt to a minimal repro, then grow it back.
The debugging funnel.
In production
- On-call sees a spike of 400s; error bodies reveal a schema change, and a client cache is refreshed without provider involvement.
- A 500 burst from one provider is correlated via request IDs to a gateway region, not the codebase.
- A latency regression breaks down as connect+TTFT growing — pointing at the gateway pool, not the model.
Best practices
- Log status, request id, latency and a redacted body summary for every model call.
- Keep a curl repro template per provider for fast triage.
- Correlate failures by request ID across layers (handler → gateway → provider).
- Alert on rate and type changes, not just raw failure count.
Common mistakes
- Debugging as if every failure is a code bug when most are config/auth/limits.
- Not logging the response body, so every issue needs reproduction from scratch.
- Treating a 500 from a gateway the same as from a provider — different owners.
- Chasing a timeout without checking whether a proxy silently dead-connected.
Common issues
400s pointing at a field your code doesn't touchAn SDK upgrade changed serialization (e.g. nulls or enum spelling)
Inspect the raw JSON the SDK emitted and align with the provider schema.
Timeout at 110s that always happens mid-streamProxy/provider idle timeout firing on slow tokens
Timeout only the connection phase; let active streams run, or tune the proxy idle timeout.
Intermittent 500s from one payload onlyA content-specific failure server-side (e.g. huge context)
Repro with curl, reduce context, and report with the request ID.
Debugging tips
- Start with curl and the smallest body; add fields back until it breaks.
- Compare a working vs failing request line by line (headers, body, URL).
- Check the clock: TLS and cert errors are often time-skew.
- If the SDK swallows errors, call the transport directly for one raw response.
Interview checkpoint
Quick quiz
Which signal identifies the exact transaction across provider and your logs?
A 400 error always indicates a bug on the provider's side.