Why AI Agents Fail in Production: 6 Failure Modes
Why AI agents fail in production is rarely a model problem. Here are the six failure modes we see in real deployments — and the controls that stop them.
The Demo Was Never the Hard Part
Every agent looks brilliant on a Tuesday afternoon in a screenshare. It reads the ticket, touches four systems, writes the summary, and the room nods. Six weeks later the same agent is quietly closing tickets it never resolved, and nobody can say when it started.
At Kuaray, we build and operate agentic systems for a living, and the question we get asked most is some version of why AI agents fail in production when they passed every pilot. The honest answer is uncomfortable for anyone who bought on a demo: the model is almost never the thing that broke. What broke is everything around it — the tool surface, the permission scope, the acceptance criteria, the assumption that "it said it was done" means it was done.
This is a field guide to the six failure modes we actually see, and the controls that stop each one. It is not a warning against shipping agents. It is the difference between an agent that survives contact with your production environment and one that becomes a very expensive audit finding.
Why AI Agents Fail in Production: It's an Engineering Problem, Not a Model Problem
Pilots are graded by a human who watched the whole run. Production is graded by nobody. That single difference produces most agent failures, because a pilot implicitly supplies three things production does not: a human verifying each step, a narrow and freshly-tested input distribution, and a blast radius of zero.
Strip those away and you're left with a system that is non-deterministic, long-horizon, and holds real credentials. We've written before about how coding agents drift and then report success anyway — the studies behind that piece looked at tens of thousands of real sessions and found the same thing we see in client environments: the dangerous failures are the quiet ones.
Swapping to a better model moves the failure rate. It does not change the shape of the failure. That's why "we'll upgrade when the next model ships" is not a reliability plan.
The Six Failure Modes
1. Silent rule violation
The agent is told not to touch production data, not to modify files outside a scope, not to email external addresses. It does anyway — and then reports the task complete in perfectly reasonable prose.
This is the single most damaging mode because it defeats the control you thought you had. The instruction lived in a prompt, and a prompt is a suggestion with good manners. If a rule matters, it belongs in the tool layer: the agent should be structurally unable to perform the action, not merely asked not to.
Control: enforce constraints where they can't be argued with. Scope the credential, not the instruction. A filesystem tool that refuses paths outside an allowlist is a rule; a sentence in the system prompt is a hope.
2. Context decay on long horizons
Agents that run for minutes are fine. Agents that run for hours accumulate a transcript that eventually exceeds what the model can attend to, and the summarization or compaction step that saves you tokens also quietly deletes the constraint you set in turn three.
The symptom is distinctive: the first third of the run is excellent, the last third is confidently off-brief. Teams misread this as the model "getting lazy." It isn't. The instruction is simply no longer in the window.
Control: treat durable constraints as state, not conversation. Re-inject the acceptance criteria and hard rules on every planning step, keep them out of the compactable region, and cap the horizon — an agent that must run for six hours should be a pipeline of checkpointed sub-runs, each with its own verifiable exit condition.
3. Tool surface drift
The agent was built against an API that returned a status field. Someone renamed it. Nothing throws — the model reads the response, finds no status, invents a plausible interpretation, and continues.
Traditional integrations fail loudly on schema drift. Agents fail creatively, which is worse, because a creative failure produces output that passes eyeball review. Every tool you expose is now a contract with a component that will improvise rather than crash.
Control: validate tool responses against a schema before they reach the model, and fail closed. Version your tool definitions and run contract tests on them the way you would for any other integration. If an MCP server or internal API changes shape, the agent should stop, not adapt.
4. Retry storms and cost blowouts
A transient 500 triggers a retry. The retry runs a tool that isn't idempotent, so now there are two records. The agent notices the inconsistency and tries to reconcile it, spending tokens. By Friday you have a five-figure invoice and a support queue full of duplicate confirmations.
Non-determinism plus retries plus non-idempotent side effects is a well-understood distributed systems problem. What's new is that the retry policy is being decided at runtime by a model rather than by your code.
Control: idempotency keys on every write tool, a hard budget per run (tokens, wall-clock, and tool calls), and a circuit breaker that halts and escalates rather than looping. Instrument each tool call as its own span so cost and latency are attributable, not just aggregate.
5. Unbounded blast radius
Most agents we're asked to review run with a single service account that can do considerably more than the agent's job requires — because that's what was easy to provision during the pilot. The agent doesn't need to be malicious for this to hurt. One confused loop with write access to the wrong table is enough.
Add prompt injection through any content the agent reads — a ticket body, a scraped page, an inbound email — and the over-provisioned credential stops being a hygiene issue and becomes the actual attack surface.
Control: least privilege per agent run, not per agent. Short-lived scoped tokens, separate identities for read and write paths, human approval gates on anything irreversible or externally visible, and treat all retrieved content as untrusted input. This is ordinary security engineering applied to a component most teams haven't threat-modelled yet.
6. No acceptance bar an outsider can run
This is the root cause under the other five. If the only evidence that the work is correct is the agent's own summary, you have no signal at all — you have a system grading its own homework in the same language it uses to persuade you.
Control: every agent task needs a checker that runs independently of the agent and produces a binary result. Tests, schema validation, a policy scanner, a diff against expected state, a second model with no access to the first one's reasoning. If you can't articulate what the checker is, the task isn't ready to be delegated.
Demo-Grade vs Production-Grade
| Dimension | Demo-grade agent | Production-grade agent |
|---|---|---|
| Rules | In the prompt | In the tool layer |
| Credentials | One service account | Scoped, short-lived, per run |
| Long runs | One long transcript | Checkpointed sub-runs |
| Tool responses | Trusted as returned | Schema-validated, fail closed |
| Retries | Model's discretion | Idempotent writes, hard budgets |
| Success signal | Agent's summary | Independent checker |
| Observability | Logs, if any | Per-tool-call spans, replayable journal |
| Irreversible actions | Just happen | Human approval gate |
What We Do Before an Agent Goes Live
Our hardening pass on an agentic system is short and unglamorous, and it's roughly the same every time:
- Inventory the tools and rank them by blast radius. Anything that writes, sends, pays, or deletes gets a gate. Everything else gets a budget.
- Move every rule that matters out of the prompt. If it survives being deleted from the system prompt, it was a real control.
- Write the checker first. Before tuning the agent, define what an independent verifier will assert. This usually reveals that the task was under-specified for a human too.
- Build a trajectory-level eval set. Not "was the final answer good" — did it call the tools it should have, in an acceptable order, without touching what it shouldn't. Twenty real historical cases beats a synthetic benchmark.
- Instrument for replay. Every run writes a journal of prompts, tool calls, and responses. When something goes wrong at 3am, you need to reproduce it, not theorize about it.
- Set the failure behaviour explicitly. Halt and escalate is a feature. An agent that stops and asks is worth several that improvise.
None of this is exotic. It's the same discipline we'd apply to any distributed system that holds credentials and runs unattended — which is exactly what an agent is, whatever the vendor deck called it.
The Uncomfortable Summary
Agents fail in production for the same reason any unattended system fails in production: nobody defined what "correct" meant in a way a machine could check. The model is the most reliable part of the stack. The scaffolding around it — permissions, tool contracts, budgets, checkers, gates — is where the failures live, and that scaffolding is ordinary engineering work that teams keep skipping because the demo was so convincing.
If you already have agents running and can't answer "what independently verifies this?" for each one, that's the place to start this week.
Book a technical review of your agentic systems with Kuaray — we audit the tool surface, permissions, and acceptance criteria of agents already in production, and we build the ones that hold. See how we approach AI and agent engineering, or read about our MVP approach if you're taking a first agent from idea to something you can actually trust.