Why AI Agent Costs Are So Hard to Predict

Estimating the cost of a chatbot feature is straightforward: average tokens per message, times messages per day, times price per token. Estimating the cost of an agent is not, because an agent isn't one call — it's a loop, and the number of iterations in that loop isn't fixed. The same task can resolve in two tool calls on one run and fifteen on another, depending on what the model decides to try, what comes back from a tool, and how many times it decides to double-check itself.

The loop is the cost driver, not the task

A support-ticket-triage agent and a code-review agent look similar on paper — both are 'one task in, one result out' from the user's point of view. But under the hood, each step in the reasoning loop is its own billed API call, and the model itself decides how many steps to take. A task that should take three tool calls can spiral to twelve if the agent gets an ambiguous tool response and decides to re-verify, or if it's stuck in a retrieval-refine-retrieval pattern trying to find the right piece of context.

Context grows every iteration

Each step in an agent loop typically re-sends the full conversation and tool-call history so far, not just the new message. That means input token cost doesn't stay flat across iterations — it compounds. A 10-step agent run isn't roughly 10x the cost of a single call; it's often significantly more, because step 10 is carrying the token weight of steps 1 through 9 in its context window.

What to track instead of a per-call estimate

• Cost per completed task, not cost per API call — group every call in a single agent run under one task ID so you can see the total, not just the average step.
• Iteration count distribution — knowing that 90% of runs finish in 4 steps but 10% run to 20+ steps tells you where a loop guard would actually help.
• Cost variance, not just average cost — a task that averages $0.08 but occasionally spikes to $4 needs a different fix than one that's steadily $0.08 every time.
• A hard step-count or spend ceiling per agent run — the single most effective guard against a reasoning loop that never terminates on its own.

Set a ceiling before you optimize anything else

It's tempting to jump straight to prompt-level optimization — shorter tool descriptions, tighter system prompts, fewer available tools. Those help, but they don't fix the tail risk: the one run in a thousand that gets stuck in a loop and burns through a budget meant for the other 999. A per-run cap that force-stops an agent after N steps or $X, whichever comes first, is the cheapest insurance available and should exist before any prompt tuning.

Agent-based features are usually the least predictable line item on an AI bill, and the least visible one in a per-key or per-day view — a spike buried inside a handful of long-running tasks doesn't show up the same way a burst of chatbot traffic does. Tracking cost at the task level, not just the call level, is what makes that spike visible before it's a pattern.