Function Calling
Intermediate 12 min read Function Calling

Error Handling & Retries

Failure is part of the loop

Tool calls fail: timeouts, bad args, service errors, rate limits. A typed error envelope returned to the model continues the loop gracefully; robust retry logic with backoff keeps agents working under transient failures.

errorsretrybackoffenvelopetimeoutrate limit
Prerequisites: Tool Dispatch
Before we start

What you will learn

  • Return typed error envelopes to the model
  • Implement backoff retries for transients
  • Distinguish retryable vs permanent errors
Introduction

What is this?

Error handling in function calling means errors stay INSIDE the loop: a tool that fails returns a structured envelope describing what failed — so the model can recover or report honestly. Retry logic applies to transient failures (rate limits, timeouts) with exponential backoff; permanent errors (bad args, auth) surface and restructure instead.

Motivation

Why do you need it?

  • Silent 'empty result' failures poison the model's reasoning.
  • Typed envelopes let the model self-correct or fail honestly.
  • Retry with backoff absorbs provider throttling without spikes.
  • Error taxonomy (transient vs permanent) is the backbone of the retry policy.
Think about it

Mental model

Imagine

Kitchen ticket with a '86'd' note

Think of

Typed error envelope

When the kitchen is out of a dish, the waiter is told '86 short ribs', not silence. Back to the guest: apologize, offer the alternative (model recovers), or flag the table honestly. Silence would mean the guest waits forever — exactly what silent failures do to a model loop.

Mechanics

How it works

  • Wrap each tool execution in try/catch (plus timeout).
  • Format errors: code, message, optional details — as a tool result envelope.
  • Let the model retry on transient, adapt on permanent.
  • Apply backoff (e.g. 1s, 2s, 4s) for external rate limits.
  • Count retries; after cap, return the honest failure to the user.

Error flow.

Tool call
Success?
Transient
Permanent
Model recovers
Hands-on

Examples

Envelope + backoff

Envelope + backoff
function run(tool) {
  for (let attempt = 1; attempt <= 3; attempt++) {
    try {
      return { ok: true, result: tool() };      // success
    } catch (e) {
      const transient = /rate|timeout|429|503/.test(e.message);
      if (transient && attempt < 3) { wait(2 ** attempt * 1000); continue; }
      return { ok: false, error: { code: "tool_error", message: e.message,
                                    transient } };
    }
  }
}

Backoff retry for transients; permanent errors as net envelopes.

Real world

In production

  • Read/write wrappers return envelopes so the loop self-heals.
  • Rate-limited providers get backoff before failure reaches the model.
  • Eval harness asserts graceful degradation on injected errors.
Guidelines

Best practices

  • Distinguish transient vs permanent by error code.
  • Never expose raw stack traces to the model.
  • Cap retries AND aggregate time.
  • Send the model a short, useful reason, not a wall of text.
Watch out

Common mistakes

  • Retrying permanent errors forever.
  • Returning 'no data' on exception (the model reasons off a lie).
  • Ignoring rate-limit headers so retries collide harder.
Root cause analysis

Common issues

Error
Rate limit storm
Root cause

Backoff missing

Fix

Exponential backoff + jitter; honor Retry-After.

Error
Model retries a bad schema call endlessly
Root cause

Permanent error unmarked

Fix

Tag permanent errors; tell the model to adapt, not repeat.

When things break

Debugging tips

  • Same tool keeps failing → check it's not a permanent error mislabeled.
  • Retries amplify load → add jitter + respect Retry-After.
  • Model confused after failure → envelope too vague; add concrete reason.
Interviews

Interview checkpoint

Practice

Hands-on lab

Retry Malformed Tool Calls
25 min · intermediate
In progress
Objectives
  • Raise a typed error for invalid tool arguments
  • Coerce and retry a malformed call exactly once
  • Cap retries so a stuck loop gives up cleanly

Retry Malformed Tool Calls

25 min · intermediate

0/3 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

A rate-limit error is usually…

2

An empty result on failure beats an error envelope.

Go deeper

Official documentation

Rate limits (OpenAI)
docs