APIs
Advanced 15 min read APIs

Designing APIs for AI

Contracts that survive model churn

Your model-facing API is a contract: predictable endpoints, typed request/response schemas, versioning, and streaming where latency matters. Design it like you're shipping to strangers.

API designcontractsversioningschemasstreamingOpenAPI
Prerequisites: Debugging API Errors
Before we start

What you will learn

  • Design endpoints by responsibility, not by model feature
  • Keep response schemas stable across model upgrades
  • Version breaking changes and document with OpenAPI
Introduction

What is this?

APIs for AI services are contracts between your app and the world (or your own product teams). The rules are classic API design — clear resources, typed schemas, explicit errors, versioning — plus an AI twist: responses must stay stable even as you swap models underneath. Design the contract to be model-independent, and the model becomes an implementation detail.

Motivation

Why do you need it?

  • A frontend coupled to your prompt format breaks the day you change a system prompt.
  • Versioned, documented APIs let teams upgrade without a big-bang deploy.
  • Typed schemas plus JSON validation catch drift before it reaches the model.
  • Streaming endpoints double as a UX lever and a latency strategy.
Think about it

Mental model

Imagine

A train timetable

Think of

An API contract

Passengers rely on the timetable, not on the engines. Trains can change underneath all day, but departure times and platforms (the contract) stay stable. Model swaps are engine changes.

Mechanics

How it works

  • Design endpoints around user intentions: /chat, /summarize, /search, /agents/run.
  • Requests carry explicit params (model/engine choice, temperature, structured schema hints).
  • Responses return only contract fields — never raw model output you can't guarantee.
  • Version the API (v1, v2) so breaking shape changes don't strand consumers.
  • Publish an OpenAPI spec; generate clients and validate requests against it.

Stable contract, swappable model.

Client
Contract layer
Service logic
Model adapter
Response
Hands-on

Examples

A contract-shaped /chat response

A contract-shaped /chat response
// POST /v1/chat
// { "messages": [...], "model": "auto" }
// 200 → this envelope, stable across model swaps
{
  "id": "chat_ab12",
  "created": 1777000000,
  "content": "the assistant's reply",
  "sources": [{ "chunkId": "c_1", "href": "/chunks/c_1" }],
  "usage": { "inputTokens": 12, "outputTokens": 8 },
  "meta": { "modelUsed": "internal-name" }
}

Consumers depend on id, content, sources, usage — never on raw model quirks. `modelUsed` is metadata, not contract. Swapping upstream models doesn't break the schema.

Real world

In production

  • A product team ships against /v1/chat while the platform team experiments with new models behind it, version-gated.
  • An eval harness validates that responses conform to the OpenAPI schema before release.
  • Two models power one endpoint — a cheap one for fast paths, a frontier one for complex queries — with identical response shapes.
Guidelines

Best practices

  • Design schemas first; models conform to them, never the reverse.
  • Add both `/stream` and buffered variants where latency matters.
  • Version breaking changes; additive fields are the default compatibility mode.
  • Validate requests and responses with JSON Schema in your contract layer.
Watch out

Common mistakes

  • Exposing raw model output (prompt architecture leaks to clients).
  • Single unversioned endpoint that hard-breaks consumers on any tweak.
  • Returning `500` with no body for expected failures like bad input.
  • Letting internal model names and prompt structures leak into the public schema.
Root cause analysis

Common issues

Error
Frontend breaks every time the prompt changes
Root cause

Response shape depends on prompt wording instead of a contract layer

Fix

Wrap model output in a parser/validator that emits a stable schema.

Error
Consumers can't upgrade because of breaking changes
Root cause

No versioning; all clients coupled to one shape

Fix

Introduce /v2, deprecate /v1 with a window, and document diffs.

Error
Contract tests flake because 'model output varies'
Root cause

Tests assert on raw text instead of validated structure

Fix

Assert on schema validity and the stable fields, not generative text.

When things break

Debugging tips

  • Contract breakages surface first in clients: check response fields against the last OpenAPI diff.
  • If a client sees model quirks, your response layer is leaking the model's shape — normalize at it.
  • Stream vs buffer divergences — write contract tests asserting identical payload semantics.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

What is the best practice for response shape across model swaps?

2

Adding an optional field to a response is usually a breaking change.

Go deeper

Official documentation

OpenAPI specification
spec
JSON Schema
spec