An agent is a loop. The model picks a tool, something runs it, the result goes back into the conversation, and the model is asked again. The ReAct paper named the pattern in 2022, every framework implements a version of it, and the loop is the reason agents can do useful multi-step work.
It is also the reason they are expensive in a way that surprises people. Each step sends the entire conversation so far, including every tool result the loop has collected. Step one sends a prompt. Step twenty sends a prompt plus nineteen tool results. Nobody wrote code that costs more each time it runs; it is the shape of the loop.
Why the curve bends
Take a support-and-research agent: a 4,000-token base prompt (instructions plus tool definitions), 1,500 tokens of tool result per step, 300 tokens of output per step. Nothing unusual.
The context in front of step k is the base prompt plus everything the previous k−1 steps produced. So the tokens read across a whole task are:
tokens read = S·base + (tool + output) · S(S−1)/2
S = 20, base 4,000, tool 1,500, output 300:
20 × 4,000 + 1,800 × 190 = 422,000 input tokens for one taskThat second term is the problem. It is quadratic in the number of steps, and it belongs entirely to text the model has already read.
Step 20 of this task reads 38,200 tokens; step 40 reads 74,200. The work the agent is doing at step 40 is no larger than at step 1, but the call costs about eighteen times more.
What that costs
At Claude Sonnet 5 list prices, with full history and no caching:
| Step limit | Cost per task | Time at 3.5s a step |
|---|---|---|
| 5 steps | $0.091 | 18s |
| 10 steps | $0.272 | 35s |
| 20 steps | $0.904 | 70s |
| 30 steps | $1.896 | 105s |
| 40 steps | $3.248 | 140s |
At a thousand tasks a day, a 20-step limit is $904 a day of model spend, about $27,000 a month. The same feature capped at ten steps is $8,200. Nothing about the agent's capability changed between those two numbers except how long it is allowed to wander.
quick check
An agent task with a 20-step limit costs about $0.90. You raise the limit to 40 steps because a few hard tasks need it. What happens to a task that uses all 40 steps?
Each step resends the whole history, so total tokens grow with the square of the step count. Twenty extra steps are also twenty extra copies of everything the first twenty collected: $0.904 becomes $3.248.
Latency is the other budget
Seventy seconds is a long time to watch a spinner, and the step count is usually the reason. Anthropic's Building effective agents puts the trade plainly: "agentic systems often trade latency and cost for better task performance, and you should consider when this tradeoff makes sense".
Latency also compounds differently from cost. Cost is dominated by the last steps, because they carry the most context. Latency is dominated by the number of steps, because each one is a full round trip: model call, tool call, back again. Cutting tool results in half barely moves the clock; cutting the step count in half nearly halves it.
If a human is waiting, budget the steps like you would budget milliseconds in a voice pipeline: decide the total first, then see how many round trips fit inside it. If nobody is waiting, run the task in the background, tell the user when it is done, and stop paying for interactive latency you are not using.
Work out your own loop
The defaults are the agent above. Change the tool result size first; it is usually the number people have never measured.
calculator · list prices per 1M tokens, checked 12 Sep 2026
What one agent task costs, step by step
cost of each step, as the history grows
cost per task, same loop, four ways to handle history
Assumes every step resends the conversation so far, which is how a tool-use loop works unless you edit the history. With caching on, the context that existed at the previous step is priced at the provider's cache-read rate and the new tokens at 1.25x input, which matches Anthropic's 5-minute cache and OpenAI's current models. Summarising costs an extra model call, which is counted as output at the summary size.
Four ways to flatten the curve
1. Cache the history. Each step's prompt is the previous step's prompt plus a bit. That is the ideal shape for a prompt cache: the same 20-step task costs $0.217 instead of $0.904, and the model sees exactly what it saw before. Keep the loop append-only so the cached prefix survives, and put stable tool definitions and instructions at the very front.
2. Do not let big results into the history. A tool that returns a 40-page document turns every later step into a 40-page prompt. Truncate at the tool boundary, return the fields the agent needs rather than the whole payload, and write large results to a file or key-value store that the agent can read back deliberately. This is the cheapest fix available and the one most often skipped.
3. Clear old tool results. Anthropic's context editing does this server-side: clear_tool_uses_20250919 clears the oldest tool results once the context passes a threshold, 100,000 input tokens by default, keeping the three most recent tool uses. Clearing invalidates the cached prefix, which is why the same API has a clear_at_least setting: only clear when you will clear enough to be worth the re-write.
4. Summarise, then start again. Anthropic calls this compaction: "taking a conversation nearing the context window limit, summarizing its contents, and reinitiating a new context window with the summary". Without caching, summarising every eight steps takes the 20-step task from $0.904 to $0.485. It costs an extra model call each time, and it loses detail, which is fine for a long research run and dangerous for a task where a detail from step three decides step nineteen.
Options 2 to 4 all make the same trade: less context for less money. It is often a good trade for a second reason, since recall degrades as the window fills. Anthropic describes that as context rot, where "as the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases". A shorter, curated history can score better as well as cost less.
Caps that actually stop things
Every agent framework ships a step limit because everyone learns this lesson. The OpenAI Agents SDK defaults max_turns to 10 and raises MaxTurnsExceeded past it. Anthropic's guidance is the same: include "stopping conditions (such as a maximum number of iterations) to maintain control".
A step limit alone is not enough. Four caps worth having, in the order I add them:
- Steps per task. Start low, at something like 10, and raise it with evidence. Log how many steps real tasks take; the distribution usually has a long, useless tail.
- Tokens per task. A budget the loop decrements as it goes, checked before each call. Steps are a poor proxy when one tool returns a megabyte.
- Wall-clock per task. Protects against a slow tool rather than a chatty model, and gives you something to show the user when it trips.
- Cost per task and per tenant. The one that matters commercially. A task that hits the ceiling should fail visibly and hand back what it has, not retry forever. The same reasoning as backpressure in a queue: an unbounded loop is an outage with a delay on it.
What should happen when a cap trips is a product decision, not an error handler. Return the work so far, say what is missing, and offer to continue. Silently truncating at step ten and answering anyway is how an agent gets a reputation for confident nonsense.
What to log
Per step: the step number, input tokens, cached tokens, output tokens, which tool ran, how long it took, and the cost. Per task: total steps, total cost, whether a cap fired and which one.
With that, three questions become answerable in an afternoon: which tool returns the results that bloat context, what share of tasks finish in under five steps, and which tenant is spending your margin. Without it, you are looking at one number on an invoice. The rest of that log line is in LLM observability in production, and the per-user arithmetic in LLM cost per user.
questions people ask
Why do AI agents cost so much more than a single model call?
Because every step resends the conversation so far. Total tokens grow with roughly the square of the step count, so a 20-step task can send 400,000 input tokens even though each individual step is small.
How many steps should an agent be allowed?
Start at around 10 and raise it only where the data shows real tasks need more. The OpenAI Agents SDK defaults to 10 turns; Anthropic recommends a maximum-iteration stopping condition as standard practice.
Does prompt caching help agents?
Yes, more than almost anywhere else, because each step's prompt extends the last one. In the worked example it takes a 20-step task from $0.90 to $0.22 with no change to what the model sees.
Is it better to summarise or truncate agent history?
Truncating is cheaper and blunter; summarising keeps more of the thread for the price of an extra model call. Both discard information, so cap the raw size of tool results first, which costs nothing in capability.
How do I stop an agent looping forever?
Cap steps, tokens, wall-clock time and cost per task, and make the failure visible: return the partial work with a note about what is missing rather than retrying silently.
How long should an agent task take?
If a person is waiting, budget backwards from what they will tolerate: at three to four seconds a step, ten steps is already most of a minute. Longer tasks belong in the background with a notification at the end.
The short version
An agent's cost is not steps times the cost of a step. Each step resends everything before it, so the total grows with the square of the step count: 20 steps cost $0.90 in the example above, 40 cost $3.25.
Turn on prompt caching first, because it changes the bill without changing what the model sees. Then stop oversized tool results from entering the history at all, and only then reach for clearing or summarising, which trade context for money.
Finally, cap steps, tokens, time and cost per task, and make a cap that trips visible to the user. Your average task was never the problem; the problem is the one that quietly ran to step forty.