Each step resends everything
Model APIs don't remember anything between calls. To continue, an agent sends the system prompt, the tool definitions and everything that has happened so far, then gets one more step back. That step's tool result joins the history, and the next call sends all of it again.
So the input isn't a flat cost per step. Step 1 sends the starting context. Step 100 sends the starting context plus 99 steps of history. Added up, the total grows with the square of the number of steps.
A worked example
Take a modest agent: 8,000 tokens to start, 1,500 more each step, 500 tokens written each step, and a 200,000-token context limit, at round example prices of $3 per million input tokens and $15 per million output tokens. These are illustrative numbers, so put your own in the agent loop cost calculator.
| Steps | Input tokens sent | Cost of the last step | Total cost |
|---|---|---|---|
| 10 | 147,500 | $0.07 | $0.52 |
| 30 | 892,500 | $0.16 | $2.90 |
| 100 | 8,225,000 | $0.48 | $25.43 |
| 300 | 47,616,000 | $0.61 | $145.10 |
| 1,000 | 187,616,000 | $0.61 | $570.35 |
| 3,000 | 587,616,000 | $0.61 | $1,785.35 |
From 30 steps to 300 is ten times the work and about 50 times the cost. Beyond that the context limit holds each step flat at $0.61, and the growth becomes a straight line. That's a much steeper line than the one you started on: at step 10 a step cost $0.07.
Time is the real multiplier
Nobody decides to run an agent for thousands of steps. It happens because nobody is watching. At one step every 20 seconds, the example agent makes 4,320 steps in a day, which costs $2,587.25. Over a weekend that's $5,211.65, and that's one agent. Run ten copies and multiply by ten.
Stories of five-figure agent bills circulate widely. We haven't been able to trace the ones we looked at to a primary post-mortem, so we don't repeat their figures here. The arithmetic makes the point without them.
Why agents loop at all
- A tool keeps failing, and the agent keeps trying again with small changes.
- A goal has no clear end, so the agent never decides it's finished.
- Two agents hand work back and forth, each expecting the other to finish.
- A check fails each time and the agent keeps trying to fix it, without the access or information it needs.
- Retry logic wraps the agent, so a run that failed is started again from the top.
What limits the cost, and what doesn't
| Control | Effect on a loop |
|---|---|
| Maximum steps per run | Stops one run. Doesn't help when the loop is outside the run, as with an outer retry. |
| Rate limits | Slow the spending but don't cap it. A slow loop still runs all night. |
| Trimming the context | Flattens the curve, which matters a lot, but each step still costs money. |
| Prompt caching | Cuts the cost of resent input. Anthropic bills cache reads at 0.1 times the base input price on many models, with lower rates on some. Cache writes cost extra, and a cached entry lasts five minutes by default, refreshed each time it's used. |
| A spend alert | Tells you. It doesn't stop anything, and it may arrive while you're asleep. |
| A budget checked before each call | Stops the spending at a number you chose. This is the control that bounds the total. |
Use several of these. Step limits and context trimming make loops cheaper, and a budget sets the most any loop can cost. See how to set a budget for an AI agent.
Spend is in early access: you can set budgets and see what each call cost today, and budgets that stop calls are being switched on across the platform.
Questions
- Why does an AI agent cost more the longer it runs?
- The model keeps no memory between calls, so the agent resends its whole history on each step. Input grows with every step, so total cost grows faster than the number of steps until the context limit holds it flat.
- How do I stop an AI agent from looping forever?
- Combine a maximum step count per run, clear end conditions, a limit on retries, and a budget checked before each call. The budget is what bounds the total cost, whatever caused the loop.
- Does prompt caching stop runaway costs?
- It reduces them but doesn't cap them. Cached input is much cheaper to resend, but every step still costs money, so a loop without a budget keeps spending.