The Annual Cost of Running AI Agents in Production

The Cheapest Line Item Gets the Most Attention

Teams can estimate token spend in an afternoon. Usage is visible, prices are public, and the arithmetic fits in a spreadsheet. So the model bill becomes the cost of the agent.

It isn't.

A production agent needs integrations that stay working, evaluations that catch changed behavior, and people who handle the cases it can't finish. Those costs don't arrive in one invoice. They sit across engineering budgets and operating teams, which makes them easy to ignore during approval.

Our position is that an annual agent budget should be built around accepted work, not model calls. I’d change my mind if inference spend consistently dominated the total cost of operated workflows. For many internal agents, the expensive part is everything required to make a generated action trustworthy.

Price the Unit the Business Receives

A model call isn't an outcome. Neither is a completed agent run if the result gets rejected. Choose a unit that the business already recognizes: an invoice reconciled, a case prepared for decision, or a catalog change accepted into the system of record.

Now count what it takes to produce one accepted unit. Include calls made during retries and time spent on escalation. If an operator corrects the output before it can be used, that labor belongs in the unit cost.

This denominator prevents easy self-deception. A workflow may have low infrastructure spend while producing little usable work. Another may use an expensive model but finish high-value cases with almost no repair.

Keep volume assumptions explicit. Annual projections often multiply a perfect demo by the whole queue. Real throughput is constrained by review capacity, source availability, and the cases that remain out of scope.

Model and Compute Costs Move With Behavior

Inference cost depends on more than the selected model. Context can grow as the agent gathers documents. Failed calls may be retried. Long outputs can feed another step, causing the same text to be processed again.

Record usage by workflow stage. A single monthly total won't tell you whether retrieval, planning, or repair is driving spend. It also won't show that one request type produces repeated loops.

Caching can help when the same stable context is read often. Smaller models may handle routing or extraction. But cost changes need evaluation. A cheaper call that increases human corrections can raise the accepted-unit cost.

Capacity matters too. Some workloads are steady, while others arrive near a reporting deadline. Budget for the service level the operation needs rather than an annual average that hides peaks.

Human Review Is a Real Operating Cost

Human oversight is often described as a safety feature and left unpriced. Every escalation consumes attention. Reviewers need enough context to decide, and complex cases may bounce back to the agent.

Measure minutes per review and the share of work that enters the queue. Separate routine approval from genuine repair. A fast approval on nearly every case may indicate that the gate should become policy-based. Frequent correction points to a quality or scope problem.

Don't value reviewer time at zero because the people already work there. The queue displaces other work and can add delay to the process. That delay may be the larger cost.

Training belongs in the budget as well. Reviewers need to understand the action boundary, reason codes, and escalation path. When policy changes, that knowledge has to change with it.

Engineering Ownership Doesn't End at Launch

An agent depends on systems owned by other teams. APIs change. Authentication expires. A source adds a field with an unexpected meaning. Model providers update behavior.

Someone must triage alerts and maintain the evaluation set. Someone must decide whether a failure is safe to retry. Production ownership is recurring work, even when the agent code stays still.

Estimate named engineering capacity instead of adding a vague maintenance percentage. The need may be small for a narrow workflow, but it isn't zero. Include time from security or data owners when they routinely participate in changes.

Shared platform costs should be allocated without pretending the split is exact. Observability, secret management, and agent infrastructure may support several workflows. Pick a consistent allocation rule, state it, and update it when usage changes enough to alter decisions.

Controls and Evidence Have Storage Costs

Consequential agents need an operating record. Logs capture requests and actions. Evaluation runs retain examples and scores. Approval packets may need to remain available for audit.

Storage is only one part. Sensitive records need access controls and retention policies. Investigation interfaces take engineering work. Redaction may be necessary before a trace can enter an evaluation set.

Don't retain everything forever because it might be useful. Decide which observable events are needed to explain an action and meet policy. Hidden reasoning isn't a substitute for evidence. Source versions, requests, approvals, and resulting state are far more useful in an incident.

The cost model should include periodic control tests. A kill switch that nobody exercises may fail when needed. An approval rule can drift from the action it was meant to govern.

Failure Changes the Economics

Expected operating cost doesn't capture the full decision. Some failures create rework. Others create a customer obligation or a payment that must be recovered. The budget needs a view of consequence, even when the exact probability is uncertain.

Use scenarios tied to actual action classes. What happens if the agent sends a duplicate request? What if it uses a stale price? Describe the recovery path and who does the work.

This isn't an invitation to invent a dollar figure for every risk. Mark uncertainty honestly. A bounded scenario with a named response is more useful than false precision.

If the downside can't be tolerated, narrow the agent's authority or keep a person in the execution path. Cost savings don't repair an action the organization was never willing to accept.

Build the Annual View From Production Evidence

Start with a controlled release and collect accepted-unit economics. Measure model use, review time, failed integrations, and engineering interventions. Project from that record, with separate assumptions for volume growth and scope changes.

Revisit the model after material changes. Adding a new action can alter review demand. Moving to a cheaper model can change repair rates. A new buyer-facing interface may add transaction revenue that an internal savings model never considered.

The right comparison isn't agent cost versus one employee's salary. Compare the full operated workflow before and after, including delay and the work that still requires judgment. That's the budget an engineering leader can defend a year later.