The cost nobody budgets for
Teams estimate agent cost by looking at the prompt, multiplying by the price per token, and getting a number that turns out to be wrong by an order of magnitude. The reason is structural: an agent does not send its context once. It sends the entire accumulated transcript on every single turn.
Work through what that means. Each pass through the loop adds the model's action and the tool's result to the conversation. On the next pass, all of it is re-sent - because a language model has no memory between calls, and the transcript is the memory. If each step adds roughly the same amount of material, the total input billed across N steps is not N times the step size, it is N(N+1)/2 times it.
Concretely: twelve steps, each adding about 5,000 tokens of tool output and reasoning. The final turn sends around 60,000 tokens of input. The run bills around 390,000, because every earlier turn paid for everything before it. Same task, same model, more than six times the number a naive estimate gives.
Agent cost grows with the square of run length, not linearly with it.- the single most useful fact for anyone budgeting one
The mitigation that matters most here is prompt caching, offered by every major provider: the stable prefix of the conversation is stored server-side and re-read at a fraction of the normal input price. It does not change the shape of the curve, but it substantially lowers the constant, and an agent built without it is paying full price to re-read its own history dozens of times.
Where the money goes
Four line items account for nearly all of it, and the ranking surprises people.
- Re-sent context - almost always the largest. See above. It is not a line on any invoice, which is exactly why it goes unexamined.
- Tool output. The biggest driver of how fast context grows. A tool returning a full document instead of the relevant three lines does not cost you once - it costs you on every subsequent turn of the run.
- Tool definitions and system prompt. Re-sent every turn too. Thirty tool schemas is a meaningful fixed tax on each pass, which is one more argument for the narrower agents described in Multi-agent systems.
- Output and reasoning tokens. Priced several times higher than input at every provider, but generated in far smaller volumes - so they usually matter less than the number on the price sheet suggests. The exception is extended reasoning, where a model may think for thousands of tokens before acting, on every step.
The practical consequence: the highest-leverage cost work is almost never prompt trimming. It is making tools return less, and making runs shorter.
Paying twice for failure
A failed run is not free. It is a full-price run that produced nothing, and any honest cost model has to carry it.
Three specific drains, all described in Why agents fail and all with a price attached here:
- The non-terminating loop. The most expensive single failure available, because it burns the entire iteration budget at the widest part of the context curve, where every turn is at its most expensive. A hard cap is a cost control before it is anything else.
- Retries after a bad step. Recovery is usually more expensive than the original attempt: the failed attempt is still in the transcript, so every retry is paying to re-read the mistake.
- Silent partial success. The cheapest failure on the invoice and the most expensive overall. You pay once for the run and again for the human who finds and fixes what it missed.
Watch the 95th percentile, not the median. Agent cost distributions have long tails - most runs are unremarkable and a small number are ruinous - and a median that looks fine can hide a tail that dominates the monthly bill.
The levers that actually work
Roughly in order of how much they return for the effort:
- Shorten the run. Because of the quadratic, halving the steps cuts cost by around three quarters. Nothing else comes close. Splitting one long run into two shorter ones, each with its own clean context, is a cost decision as much as a reliability one.
- Make tools return less. A path instead of a file, a count instead of the rows, the matching lines instead of the log. Every token a tool does not return is a token you do not pay for again on each later turn.
- Turn on prompt caching. Structure the conversation so the stable part - system prompt, tool definitions, task description - sits at the front and stays byte-identical, then let the provider cache it.
- Match the model to the step. Planning and final synthesis may need the strongest model. Classifying a result or extracting a field does not. Mixed-model pipelines are one of the larger savings available and one of the least used.
- Cap context explicitly. Summarise or drop old turns at a threshold rather than letting the window fill. This trades some quality for a bounded bill, and the trade is usually worth making.
Cost per outcome, not per token
Token price is an input, not a metric. The number worth tracking is cost per successful outcome - total spend, including everything that failed, divided by the work actually delivered.
The arithmetic is unforgiving in a useful way. If runs average $0.40 and 80% succeed, each success costs $0.50, because the failures are carried by the successes. At a 50% pass rate, each success costs $0.80 - the same agent, twice the unit cost, purely from reliability. This is the point where cost work and the measurement work in How to evaluate an agent turn out to be the same project: improving the pass rate is often a bigger cost lever than any token optimisation.
Two refinements make the number honest. Include the human time spent reviewing and correcting output - an agent that needs ten minutes of checking per run is not cheap at any token price. And measure against the alternative, which is usually a person doing the task, sometimes a simpler script, and occasionally not doing it at all.
Works versus affordable
The uncomfortable finding for a lot of teams is that these are separate questions, and passing the first says nothing about the second. A pilot on a handful of runs a day is a rounding error; the same design at a thousand runs a day is a budget line someone has to defend.
Three checks before scaling anything:
- Multiply by real volume, at the p95 not the median. If that number is unacceptable, the design needs to change now - not after the invoice arrives.
- Compare against the thing it replaces. Including the review time. An agent at $0.50 per outcome replacing twenty minutes of skilled work is obviously worth it; the same agent replacing thirty seconds of clicking is not.
- Decide what you will do when prices move. They have trended down consistently, but a design that is only viable at next year's prices is a bet, and it should be a conscious one.
The best cost work does not feel like cost work. Shorter runs, narrower tools, tighter outputs and a higher pass rate are the same changes that make an agent reliable - which is the useful thing about this subject. Optimising for the bill and optimising for quality point in the same direction almost every time.