APIs
Beginner 12 min read APIs

HTTP & REST for AI

The protocol under every model call

Every LLM, embedding and vector API is HTTP under the hood. Method, URI, headers, body and status code — plus the REST conventions that make APIs predictable — are the foundation of all AI integration.

HTTPRESTmethodsstatus codesheadersJSON
Prerequisites: Python for AI Engineering
Before we start

What you will learn

  • Explain the building blocks of an HTTP request and response
  • Map REST verbs to actions and interpret status codes
  • Build and inspect a model API call by hand
Introduction

What is this?

HTTP is a request/response protocol: a client sends a method, URI, headers and an optional body; a server replies with a status code, headers and a body. LLM APIs are HTTP APIs that usually accept and return JSON. Understanding the wire format lets you debug any provider, proxy or gateway without guessing.

Motivation

Why do you need it?

  • Model providers are HTTP services — 'why did my call fail?' is always answerable with a status code and a response body.
  • Retries, timeouts and rate limits are HTTP semantics before they are library features.
  • Gateway and instrumentation layers rewrite and observe HTTP, so knowing it lets you read traces and logs.
  • Streaming completions are just HTTP chunked responses, not magic.
Think about it

Mental model

Imagine

Ordering at a restaurant

Think of

An HTTP request/response

You say what you want (method + URI + body), the kitchen answers with a result code and your plate (status + response body). Some answers are 'on the way' (202), some are 'Menu item gone' (404), and some are 'you can't ask that' (403).

Mechanics

How it works

  • The client builds a request: POST /v1/chat/completions with JSON body and an Authorization header.
  • The server routes by method + path and returns a status code like 200 (ok) or 429 (rate limited).
  • Headers carry auth, content type, retry-after and request ids for tracing.
  • Bodies are typically JSON; streaming uses chunked encoding with SSE (text/event-stream).
  • Gateways and providers add latency, token and error metadata in headers or the response envelope.

Anatomy of a model API call.

Client
Transport
Server
Status + body
Client decodes
Hands-on

Examples

A raw model call with curl

A raw model call with curl
curl -sS https://api.example-llm.example/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"a-frontier-chat-model","messages":[{"role":"user","content":"Hello"}]}'

Method POST, a JSON body describing the model and messages, and auth via header. The response will be JSON (or an SSE stream if stream:true).

Real world

In production

  • A retry loop inspects status 429 (respect Retry-After) and 5xx (retry with backoff) but never retries 4xx validation errors.
  • A gateway surfaces x-request-id headers to correlate every model call with logs and billing.
  • Streaming chat UI consumes an SSE stream, rendering tokens as they arrive instead of waiting for the full body.
Guidelines

Best practices

  • Never log full Authorization headers or API keys.
  • Log method, path, status and duration for every outbound model call.
  • Read error response bodies — providers explain 400s in detail.
  • Set explicit timeouts; an HTTP hang is the classic silent killer.
Watch out

Common mistakes

  • Retrying 400/401/403 as if they were transient — they won't heal with backoff.
  • Ignoring the response body on errors, then guessing why a call failed.
  • Treating 200 as success when the body carries an error envelope.
  • Reusing one Authorization header across ten probe logs, leaking the key.
Root cause analysis

Common issues

Error
400 Bad Request from a model API with no explanation
Root cause

Malformed body, unknown model name, or schema validation failure

Fix

Print the response body; it names the offending field.

Error
Slow first token, everything else fast
Root cause

TLS/proxy setup or a gateway cold path, not the model

Fix

Measure connect vs TTFT and inspect the gateway hop.

Error
Streaming responses arrive as one blob at the end
Root cause

Server (or a proxy) buffered instead of flushing chunks

Fix

Disable buffering at the proxy and verify Content-Type text/event-stream survives.

When things break

Debugging tips

  • Reproduce with curl before blaming the SDK — strip the library and see the raw exchange.
  • Check for a proxy rewriting paths: a missing /v1/ segment produces confusing 404s.
  • Compare Content-Type of request and response (JSON vs SSE) to spot streaming misconfig.
  • Log the request body hash so you can prove what you actually sent.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

Which HTTP status normally means 'rate limited'?

2

Model APIs only ever return 200 or 4xx/5xx with JSON bodies.

Go deeper

Official documentation

MDN HTTP overview
docs
IANA status codes
docs