AI Agents in Production: The Reliability Playbook

Last Updated on October 23, 2026 by Elizabeth Sramek

The agent demo worked. It read the ticket, looked up the customer, drafted a refund and posted to Slack in eleven seconds while three people watched. Six weeks later the same code is on the incident board because it refunded one order four times, and nobody can say why: the only artifact of the run is a log line reading agent completed.

That gap is not a model problem. Running AI agents in production is an operations problem, and every control that closes it already exists in ordinary workflow automation: idempotency keys, scoped credentials, retry budgets, circuit breakers, dead-letter queues, structured logs, regression tests. Teams rediscover all of it the expensive way because agent tooling is marketed as if orchestration were the hard part.

This post is the playbook: the gaps between demo and system, the control that closes each, and a readiness checklist you can fail a project on.

Why AI agents in production fail, and the short list of controls that fix most of it

Agents fail in production because a demo optimises for one happy path with a human watching, while production is thousands of unwatched runs with arbitrary inputs, provider outages and real side effects. The model is usually the least broken component in the stack.

Sort agent incidents by root cause and the same short list keeps showing up:

• Output shape drift. The model returned prose, or JSON with a renamed field, and the next node threw.
• Duplicated side effects. A step succeeded, the run crashed after it, the retry replayed everything, and the invoice went out twice.
• Excess authority. The agent held a credential far broader than the task needed, and one bad plan used it.
• Unbounded loops. No iteration cap, no token budget, so a task the agent could not finish burned money until the invoice arrived.
• Provider degradation. A 429 or a timeout with no fallback path took the whole queue down.
• No trajectory. The logs recorded the outcome but not the steps, so the failure cannot be reproduced.

None are frontier research problems. Each has a known control, and the controls are boring: validate, scope, cap, fall back, trace, gate. That is the rest of this post. For the case that some frameworks are not ready for client work at all, the CrewAI reality check covers where the abstractions leak.

The demo-to-production gap, gap by gap

Every demo makes six assumptions that production invalidates, and each maps to an engineering control rather than a better prompt.

Gap	What the demo assumes	What production requires	Concrete control
Non-determinism	The output looks right, so the format is right	Machine-parseable output on every run, including edge inputs	Schema-enforced output plus a validate-and-repair loop with a hard attempt cap
Error handling	Failures are visible to the operator watching	Failures are classified, retried or parked, and alerted on	Typed error taxonomy, exponential backoff with jitter, dead-letter queue
Cost variance	One run cost a few cents	Cost per run bounded and monitored, not just monthly spend	Per-run token budget, iteration cap, model routing by step type
Observability	You can read the console output	Any failed run from last Tuesday can be reconstructed exactly	Per-step trace with inputs, tool calls, outputs, tokens, latency, and a trace ID
Security boundary	The agent only does what you asked	The agent can only do what you granted, even under adversarial input	Least-privilege per tool, read-only default, allowlisted endpoints, approval on writes
State management	One run, one session, clean start	Concurrent runs, partial failures, safe replays	Idempotency keys on every side effect, checkpointed state, run-level locks

Nothing in the right-hand column is agent-specific; it is what you would apply to any integration that touches billing. If your existing automation failure handling is thin, an agent finds every hole in it faster than a deterministic workflow does, because it generates novel inputs to its own downstream steps.

Non-determinism is a design constraint, not a bug

Stop trying to make the model deterministic and build the system so non-determinism cannot cause damage. Temperature zero is not a guarantee, and prompting for a format is a request, not a contract. The contract lives in your code.

Enforce the schema, then validate it anyway

Use provider-side structured output where it exists. OpenAI’s Structured Outputs with strict: true constrains generation to a JSON Schema you supply, and is documented as adhering to that schema rather than merely producing valid JSON the way older JSON mode did. It also makes safety refusals programmatically detectable instead of arriving as unparseable prose.

It does not remove the need to validate. Structured output constrains shape, not semantics: a schema-valid response can still carry an order ID that does not exist or an amount outside your approved range. Validate against the schema, then against your business rules, and treat both failures the same way. The mechanics of getting parseable objects out reliably are in the guide to forcing strict JSON out of the API.

The retry-with-repair pattern

When validation fails, do not retry the identical request and hope. Feed the validation error back as context so the second attempt is a repair, not a coin flip, and cap the attempts so a permanently malformed task cannot loop.

import json, hashlib
from jsonschema import Draft202012Validator, ValidationError

REFUND_SCHEMA = {
    "type": "object",
    "additionalProperties": False,
    "required": ["order_id", "amount_cents", "reason", "confidence"],
    "properties": {
        "order_id":     {"type": "string", "pattern": "^ORD-[0-9]{8}$"},
        "amount_cents": {"type": "integer", "minimum": 1, "maximum": 50000},
        "reason":       {"type": "string", "enum": ["damaged", "late", "duplicate", "other"]},
        "confidence":   {"type": "number", "minimum": 0, "maximum": 1},
    },
}

MAX_REPAIRS = 2

def parse_with_repair(call_model, prompt, run_id):
    validator = Draft202012Validator(REFUND_SCHEMA)
    messages = [{"role": "user", "content": prompt}]

    for attempt in range(MAX_REPAIRS + 1):
        raw = call_model(messages)
        try:
            candidate = json.loads(raw)
            errors = sorted(validator.iter_errors(candidate), key=lambda e: e.path)
            if not errors:
                # Business rules run after the schema passes.
                if not order_exists(candidate["order_id"]):
                    raise ValidationError("order_id not found in system of record")
                return candidate, attempt
            raise ValidationError("; ".join(
                f"{list(e.path) or 'root'}: {e.message}" for e in errors[:5]
            ))
        except (json.JSONDecodeError, ValidationError) as err:
            log_step(run_id, "validation_failed", attempt=attempt, detail=str(err))
            if attempt == MAX_REPAIRS:
                raise AgentHardFailure(run_id, str(err))   # goes to the DLQ, not a retry
            messages += [
                {"role": "assistant", "content": raw},
                {"role": "user", "content":
                 "That response failed validation: " + str(err) +
                 " Return only a corrected JSON object matching the schema."},
            ]

def idempotency_key(run_id, step_name, payload):
    # Same run, same step, same payload => same key => the API dedupes the replay.
    body = json.dumps(payload, sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(f"{run_id}:{step_name}:{body}".encode()).hexdigest()

Idempotency, or your retries will bill twice

Every write an agent performs needs a deterministic idempotency key derived from the run, the step, and the payload, so a replayed run produces one refund rather than two. Most payment processors and a growing number of CRM APIs honour an idempotency header; where the API does not, keep your own ledger table keyed the same way and check it before the call.

This matters more for agents than for linear workflows because agents retry at the plan level. A deterministic workflow retries step seven. An agent that fails at step seven may re-plan from step one and re-execute steps two through six on the way back, repeating every side effect that is not idempotent.

The tool boundary is your security boundary

Every tool you register on an agent is authority granted to a text generator, and the blast radius is the union of what those tools can do. That is the whole security model. Prompts are not a boundary, and an instruction telling the model not to do something is a preference, not a permission.

OWASP catalogues this as LLM06: Excessive Agency, and splits the cause into three: excessive functionality (tools that do more than the task needs), excessive permissions (credentials broader than the operation), and excessive autonomy (high-impact actions executed without independent verification). Each has a matching fix.

Least privilege, per tool, not per agent

Scope every tool to the narrowest capability that completes its job, and scope the credential behind it the same way. A tool named get_customer should sit behind a read-only database role, not the key the write tools use. OWASP is explicit about preferring granular tools over open-ended ones: an agent with a run_sql tool has the authority of the database user, not of the query you imagined.

Give the agent its own identity

Do not reuse a human’s credentials. An agent running as a named service principal is auditable, revocable and rate-limitable on its own; an agent on an admin’s OAuth token is indistinguishable from that admin in every downstream log, and revoking it locks a person out of their job. Rotate its credentials like any other machine identity, covered in the ops guide to rotating API keys and the walkthrough of rotating keys inside automation platforms without breaking live runs.

Allowlist the network, do not hand over HTTP

A generic http_request tool is the most dangerous thing on most agent stacks, because it turns every other control into a suggestion: the agent can exfiltrate to any host, call any internal service reachable from your egress, and reach cloud metadata endpoints. Replace it with named tools per integration. Where a general fetch is genuinely required, enforce a domain allowlist, block private address ranges, cap response size and strip credentials from redirects.

The prompt-injection path everyone underestimates

The dangerous shape is not a user typing a jailbreak. It is an agent that fetches content, and content that instructs the agent, while the agent holds write authority. OWASP’s indirect prompt injection entry describes exactly this: text in an external source alters model behaviour when the model reads it. A support ticket, a scraped page, a PDF, a webhook body. All untrusted input the moment it enters a planning step.

Three mitigations do most of the work, and OWASP names all three: privilege control (give the application its own tokens and handle privileged functions in code rather than exposing them to the model), human approval for privileged operations, and clear segregation of untrusted external content from instructions.

Cost control is a reliability feature

An unbounded agent loop is a reliability incident and a financial one at once, which is why token budgets and iteration caps belong with timeouts rather than in the finance team’s spreadsheet. OWASP tracks the shape as LLM10: Unbounded Consumption in its 2025 Top 10 for LLM Applications.

Four controls, in order of what they save:

1. Hard iteration cap. A maximum number of plan-act cycles per run, enforced by the orchestrator, not the prompt. When it trips, the run fails loudly into the dead-letter queue.
2. Per-run token budget. Accumulate tokens across every step and abort at the ceiling. Set the ceiling from the p95 of your successful runs, not the mean.
3. Model routing by step type. Classification, extraction and routing do not need your most expensive model. Reserve that for planning and final synthesis.
4. Alert on cost per run. Monthly spend hides a regression until the invoice. Cost per run, tracked per agent version, catches a prompt change that doubled context on the day it shipped.

Worked example: how a retry loop compounds cost

Hypothetical calculation with stated assumptions, not measured data. Assume a blended 3.00 USD per million input tokens and 15.00 USD per million output tokens, and check current provider pricing before reusing the figures. Assume each step appends about 4,000 tokens, emits 600 output tokens, and resends the accumulated context, which is what most agent loops do by default.

Scenario	Billed steps	Input tokens	Output tokens	Cost per run
Naive estimate, flat context	8	32,000	4,800	0.17 USD
Reality, context accumulates	8	144,000	4,800	0.50 USD
One step fails validation 3 times	11	216,000	6,600	0.75 USD
No iteration cap, loop runs to 40 steps	40	3,280,000	24,000	10.20 USD
Same runaway across a 500-run queue	20,000	1.64 billion	12,000,000	5,100 USD

The lessons are in rows two and four. Naive per-step estimates understate cost by roughly a factor of three because they ignore quadratic context growth, and a capped loop versus an uncapped one is two orders of magnitude on a single run. Prompt caching changes these numbers where it applies, and it changes rate-limit accounting too: Anthropic documents that cache reads do not count toward input-token-per-minute limits while cache writes do.

Failure, fallback and degraded modes

Design for the provider being slow, rate-limited or gone, because all three happen and an agent amplifies each by making several calls per run. A workflow with one model call has one chance to hit a 429; an eight-step agent has eight.

Anthropic’s rate limit documentation defines limits across requests per minute, input tokens per minute and output tokens per minute, and returns a 429 with a retry-after header naming how long to wait. Honour that header: a backoff schedule your code invented is worse than the number the provider gave you. For the platform-level version, the walkthrough on handling 429 responses in Make.com covers queue shaping and retry placement.

Above the retry layer you need four things:

• Exponential backoff with jitter on transient errors, so a thousand queued runs do not retry in lockstep and recreate the outage they are recovering from.
• A circuit breaker per provider. After N consecutive failures, stop calling for a cooldown window instead of burning the queue against a dead endpoint. Half-open the breaker with a single probe request.
• Cross-provider fallback routing so a vendor incident degrades quality rather than stopping work. The mechanics, including the prompt and schema differences you have to absorb, are in the guide to LLM fallback routing between OpenAI and Anthropic.
• A dead-letter queue holding the full input, the partial trajectory and the terminal error, so a human can replay or resolve. Runs that vanish on failure are why nobody trusts the agent by month three.

Define the degraded mode explicitly. When the primary is down and the fallback is not good enough for autonomous action, route the work to a human queue with the partial analysis attached rather than retrying silently for twenty minutes.

Observability: trajectories, not averages

Aggregate metrics hide agent failures almost perfectly, so the unit of observability is the trajectory: the full ordered sequence of steps, not a success rate. A 97 percent success rate says nothing about whether the 3 percent are harmless timeouts or duplicate refunds, and a nine-second average latency hides the run that looped twelve times.

What to log on every step

Field	Why it earns its storage cost
Trace ID and step index	Stitches a multi-step run together across services and queues; without it you have fragments
Full prompt sent, including retrieved context	The only way to reproduce a bad output; a prompt template plus variables is not the same as the rendered prompt
Raw model output before parsing	Distinguishes a model error from a parser error, which are fixed in completely different places
Tool name, arguments, and result	Shows what the agent actually did versus what it said it would do
Input and output token counts	Feeds cost-per-run alerting and shows context growth per step
Latency per step and per run	Separates a slow provider from a slow tool from a long plan
Model ID, prompt version, agent version	Makes a regression attributable to a specific change instead of to vibes
Terminal status and error class	Turns failures into a countable taxonomy rather than a text search

The test is blunt: take a run that failed last week and reproduce it exactly from stored data. If you cannot rebuild the input, the retrieved context, the model version and the tool responses, you do not have observability, you have receipts.

A prompt change is a deploy

Treat prompt edits with the same ceremony as code changes, because they have the same blast radius and none of the type safety: version control, review, and a regression suite that runs before the change reaches production. Teams that keep prompts in a database field edited through an admin UI cannot answer the question “what changed on Tuesday”. The same discipline applied to workflow definitions is in the guide to version controlling n8n workflows, and it transfers directly.

Build a regression set of 30 to 100 real tasks with known-good outcomes, drawn from production traffic and weighted toward cases that have already broken. Run it on every prompt, model and tool change. Grade deterministically where you can and on a rubric where you cannot, but keep the deterministic assertions as the gate.

Then shadow-run before switching. Send production traffic to both the current and candidate configuration, execute only the current one’s side effects, and compare trajectories and cost across a few days. Shadow running surfaces the weird inputs your regression set does not have, which is where the interesting failures live.

Human-in-the-loop as architecture

Put the human checkpoint where reversal is impossible or expensive, not everywhere, because approving every action destroys the throughput that justified the agent. The question is not whether humans are in the loop, it is which actions require them.

Action class	Checkpoint	Why
Read, summarise, classify, enrich	None	Reversible, no external side effect, cheap to redo
Internal writes (draft, tag, ticket note)	Post-hoc sampling	Reversible in-house; review a sample weekly rather than each item
Customer-visible output	Approve before send, or confidence-gated	Cannot be unsent; reputational cost is asymmetric
Money movement, contracts, permissions	Always approve	Legal and financial tail; OWASP names human approval for privileged operations
Deletion or bulk mutation	Always approve, plus a row-count ceiling	Irreversible and fast; the cap is the second line of defence

The pattern that preserves throughput is confidence-gated routing: anything above the threshold executes automatically, anything below goes to a human queue with the trajectory attached. Set the threshold from your regression data and move it as evidence supports. Design the approval surface so a reviewer decides in seconds: the proposed action, the two or three facts it rests on, an approve or edit control. An approval step that needs three tabs open gets rubber-stamped within a fortnight, which is worse than no approval step because it manufactures the appearance of oversight.

The failure modes teams rediscover the expensive way

These are the mistakes that turn a working agent into an incident, with the consequence spelled out rather than called a risk.

Mistake	What it looks like in production
Retrying the whole run instead of the failed step	Non-idempotent steps two through six re-execute, producing duplicate records, duplicate emails, duplicate charges
One shared API key across agent, workflows, and humans	A rate limit tripped by the agent takes down unrelated automations, and revoking it during an incident breaks everything at once
Trusting the confidence score the model reports	Confidence is a generated token, not a calibrated probability; treat it as a routing hint validated against your regression set, never as evidence
Prompt edits made directly in production	Quality drops, nobody can identify what changed, and the rollback is somebody’s memory of the previous wording
No cap on retrieved context size	A long document blows the context window mid-run, the agent silently loses the earliest instructions, and the failure looks like a reasoning error
A generic HTTP tool left on for convenience	Every other permission control becomes advisory; internal services and metadata endpoints are reachable from a text prompt
Alerting on error rate only	Successful-but-wrong runs never alert; the agent confidently closes tickets it did not resolve and the metric looks excellent
Rolling out to 100 percent of traffic on day one	No baseline to compare against, no shadow data, and the only rollback signal is customer complaints

The production readiness checklist

Run this before shipping. Treat any unchecked item as a launch blocker or a risk someone has accepted by name.

1. Every model output feeding a downstream step is schema-validated, and failures go through a capped repair loop ending in a dead-letter queue, not an infinite retry.
2. Every write tool uses a deterministic idempotency key, and you have replayed a run to confirm it produces one side effect, not two.
3. Each tool has its own credential scoped to the narrowest capability it needs, and none shares a credential with a human user.
4. There is no open-ended HTTP or shell tool; any general fetch enforces a domain allowlist and blocks private address ranges.
5. Untrusted fetched content cannot reach a planning step that holds write authority without passing through a constrained schema first.
6. Irreversible actions require approval, and the approval surface shows the action and its supporting facts on one screen.
7. A hard iteration cap and per-run token budget are enforced by the orchestrator, and you know what happens when each trips.
8. Cost per run is tracked per agent version and alerts on a threshold, independently of monthly spend.
9. Retries use exponential backoff with jitter and honour any retry-after header the provider returns.
10. A circuit breaker and a fallback provider are configured, and you have tested the failover by blocking the primary endpoint.
11. Every step logs the fields above, and you have reproduced one historical failed run end to end from stored data.
12. Prompts, tool definitions and model IDs are in version control, and a regression set runs on every change to them.
13. The new configuration has shadow-run against production traffic long enough to see the weird inputs.
14. A documented kill switch stops new runs without losing in-flight state, and someone on call knows where it is.

Verdict: what to ship now, and what still needs a human in the path

Ship read-heavy agents now, gate write-heavy agents, and ship nothing that can move money without an approval step. That line is drawn by the cost of being wrong, not by model capability.

Ready now, with the controls above in place: research and enrichment, ticket triage and routing, document extraction into a validated schema, drafting content a human sends, internal summarisation, and code review that suggests rather than commits. These share a property: the worst outcome is wasted time, and a human sees the output before it reaches anyone who matters.

Still needs a human in the path: refunds and payments, contract and pricing changes, permission grants, bulk deletion, and any outbound customer message not reviewed by a person or gated by a confidence threshold you validated on real data. Long-horizon autonomous execution against production systems belongs here too, because multi-step plans compound error rates and a chain of ten steps at 97 percent each is not a 97 percent system.

Starting from nothing, build the boring layer first and the agent second. Put schema validation, trace logging, idempotency keys and a dead-letter queue around a single deterministic workflow, then swap one step for a model call and watch what breaks. That order costs a fortnight, and it separates an agent that survives its first bad week from one that quietly gets switched off. If you are buying an orchestration layer rather than building one, evaluate it on that list, not on how fast it produces a demo.

Frequently asked questions