Agent Reliability
Correct, safe, and honest under pressure
Reliability is defaulting to correct-and-safe even when tools fail, world surprises, or steps compound. Retries, checkpoints, verification gates, and honest failure are the levers — measured by eval coverage.
What you will learn
- Layer retries and checkpoints into loops
- Verify high-stakes outputs before acting
- Fail honestly with usable partial state
What is this?
Reliability for agents means the system handles failure as designed: transient tool errors retry with backoff, loops checkpoint so steps resume rather than restart, high-stakes actions verify before executing, and when things can't be fixed, the agent says so — with partial state and steps taken — instead of inventing success.
Why do you need it?
- Compound steps magnify single-step errors.
- Unverified claims become actions; that's the dangerous part.
- Checkpoints turn failures from restarts into resumes.
- Honesty at failure is the difference between annoying and dangerous.
Mental model
Aviation checklists
Agent reliability
Pilots don't improvise remember-if-it-feels-right; they run checklists (gates), read instruments (observations), call go-arounds (verify + retry), and declare missed approaches (honest failure) rather than push through fog to a runway that isn't there.
How it works
- Wrap every tool call: retry transients with backoff + cap.
- Checkpoint loop state per step (persistable).
- Gate consequential actions with verification (see results before next).
- Detect fictional success: require evidence artifacts.
- On unrecoverable failure, report status + partial state + next steps.
Reliability stack.
In production
- Ops agents verify rollout health before continuing the runbook.
- Migration agents checkpoint whole tenant per batch.
- Finance flows require output confirmation before submission.
Best practices
- Design for failure every step: what does THIS step do when the tool breaks?
- Persist checkpoints for long, resumable work.
- Verify side-effect claims with observable evidence.
- Track reliability metrics (retry %, failure %, honest-abort %) in evals.
Common mistakes
- Claiming done when verification didn't run.
- No resume point — a mid-task crash aborts everything.
- Honest-abort treated as a code smell instead of a feature.
Common issues
Agent reports done without artifactsTerminal unwitnessed
Require the final turn to reference concrete tool evidence.
Crash restarts everythingNo checkpoints
Persist step state; resume from last checkpoint.
Debugging tips
- Reliability metrics low → isolate: transient-fail rate vs logic-fail rate.
- Crash mid-run loses work → checkpoint per step.
- Fake-done reports → evidence gate missing at terminal.
Interview checkpoint
Quick quiz
Honest failure should…
Checkpoints let agents resume instead of restart.