Why cheaper tokens do not guarantee lower AI agent costs

Cheaper tokens do not guarantee cheaper AI agent workflows

An enterprise can spend more on AI agents even as model processing gets cheaper. Gartner predicts that token costs will fall by 95% by 2030, while inference costs, the costs of running trained models, for agentic workflows will rise more than fivefold over what its forecast calls “the next two years.” The forecasts measure different things: the price of processing and the total spending required for increasingly demanding tasks.

To understand that spending, start with the completed workflow: every action required to produce the intended result. Large language models, or LLMs, process information in units called tokens, and AI agents use these models to carry out tasks through sequences of actions. As agents take more actions and use more advanced capabilities, their consumption can grow enough to outweigh lower token prices.

Gartner calls this the “inference paradox.” Its analysts and report coauthors, Will Sommer and Sabine Zimmerhansl, argue that “the rate of innovation is outpacing the cost curve.” Gartner has a commercial interest in enterprise demand for technology research and advice, so its warnings also serve a business that helps buyers assess these decisions. The analysts’ central warning is that a roadmap can become more expensive as the work assigned to agents changes.

Autonomous execution adds work before an answer arrives

The extra work starts after a user submits a request. A basic chatbot interprets the request and generates a response, while an advanced agent may divide the problem into smaller tasks, examine intermediate results and try another approach after a failure. Each stage can require another model call or a more capable model. Sommer and Zimmerhansl report that advanced reasoning agents can already cost up to 150 times as much as basic AI chatbots for a single task.

Checking the work adds processing because an autonomous system must assess whether its output is acceptable. “They need to be able to validate their results for accuracy without necessarily having a human in the loop,” Sommer and Zimmerhansl write. If an agent finds an error, correcting it can require another cycle of model calls and checks. The system therefore consumes resources both to produce a result and to judge whether it completes the task.

These checks can also involve other agents. “They need to talk to other agents,” Sommer and Zimmerhansl add, describing systems whose agents call one another and exchange information. One action can trigger more actions elsewhere, increasing the processing charged to the original request. Gartner describes token consumption from these autonomous interactions as growing exponentially as interactions multiply.

Because these interactions can run in the background, a user’s conversations reveal only part of the activity. Sophisticated agents may keep checking outputs and responding to other agents between direct user requests. Spending can therefore accumulate while no person is waiting for a response. A budget based on visible conversations will miss some of this work.

Gartner’s comparisons show several distinct sources of cost. It reports that agents consume five to 30 times as many tokens as chatbots for equivalent tasks, while the hardware cost of training medium-sized agentic models with advanced reasoning is 2.5 times that of training simple chatbots of similar size. In a separate comparison of running trained models, Gartner puts agent inference costs at five times chatbot costs. These ratios measure consumption, training and operation separately; they are not factors to multiply automatically into the task-cost comparison.

Execution frequency determines how often an enterprise incurs these costs. Sommer and Zimmerhansl illustrate the potential scale with hundreds of agents each performing dozens or hundreds of tasks an hour. Repeated task division, validation and agent-to-agent communication can then generate substantial spending. Their example describes possible activity at scale rather than a particular enterprise deployment.

The type of work can raise spending further. Gartner points to agents handling multimodal data, information in different forms, and calling on more capable models as they divide problems into smaller tasks. These choices affect both the amount of processing and the capabilities used. A workflow’s economics therefore depend on the cost of its tokens as well as their number.

More capable workflows can also use more expensive tokens

Gartner examines these capability-related differences through its Tokenomics Model, which estimates training and inference economics across scenarios. The model combines design choices with technology improvements, hardware specifications and spending on data, infrastructure, energy and labor. With these inputs, Gartner can compare the cost of producing outputs for different kinds of work.

The design choices include the number of processing layers, LLM-as-a-judge and mixture-of-experts approaches. An LLM-as-a-judge uses a language model to evaluate outputs, while a mixture-of-experts design assigns inputs to specialized model components. Each approach changes how computation generates or assesses a result. Gartner ran 12 types of AI model with different capabilities and compared costs across workflow categories.

These figures describe modeled provider costs per token; enterprise purchase prices depend on the terms under which buyers obtain those capabilities. Gartner states that planning and learning carry eight to 10 times the provider cost per token of basic workflows. For procurement teams, this separates the economics of supplying a capability from the amount an enterprise will pay to use it.

For product owners, these differences make capability selection a decision for each stage of a workflow. Retrieving information, summarizing it and choosing an action place different demands on a model. Assigning advanced capabilities across all those stages increases exposure to their costs, so the business case must explain what the additional capability contributes to the result.

Faster responses demonstrate operational value

Higher spending can be worthwhile when autonomous execution improves service. Gartner has observed customer success agents reducing response times by 99%, giving businesses a concrete operational reason to consider them. Sommer and Zimmerhansl also say AI can carry out many routine tasks faster and more effectively than people. They add that agents can find patterns across separate systems more quickly.

These improvements still require an economic assessment of the work behind them. Faster responses can create value, but profitability depends on how that value compares with total workflow spending. “Costs will inevitably escalate, and as they do, there is no guarantee that value will grow commensurately,” Sommer and Zimmerhansl write. They describe return on investment, or ROI, as “eminently possible,” while warning that “ROI from each new generation of technology will be hard-earned.”

Manage capability, spending and success at the workflow level

To make that investment work, Gartner recommends building and maintaining complex multimodal systems with orchestration, the coordination of models, agents and their actions, to control cost and performance. This takes engineering capacity to choose capabilities and coordinate their use as tasks change. Gartner also cautions that existing systems and workflows need adaptation, making continued maintenance part of the deployment’s economics.

Without that adaptation, autonomy can create uncontrolled spending. “Defaulting to generic autonomous intelligence will result in unbounded costs orders of magnitude higher than those of optimized product ecosystems,” Sommer and Zimmerhansl warn. Their recommendation makes selective use of intelligence an engineering responsibility: teams need to determine which capabilities each task requires and constrain how the system invokes them.

Model routing offers one way to control that use. Gartner recommends sending queries to the most cost-efficient model and preventing simple tasks from calling frontier models, those at the leading edge of capability, by default. It calls the assignment of different levels of model processing to different needs “inference tiering.” In a workflow combining straightforward retrieval with demanding reasoning, teams can make that choice at each stage.

Purchasing arrangements should reflect these differences in demand. Gartner advises adopting usage-based pricing and replacing flat compute fees with tiered plans that scale with need. A plan’s financial effect depends on the work assigned to it, so buyers need to calculate how expected task volume and capability choices translate into charges under each arrangement.

Because capabilities and prices change, model selection also needs regular review. Sommer and Zimmerhansl suggest considering mandatory, continuous model-refresh cycles supported by data fine-tuning, which adapts a model using additional training data, and self-learning feedback loops, which use results to improve later behavior. These processes create an ongoing maintenance obligation. Teams need to revisit the model choices and cost assumptions made when the workflow was first designed.

That obligation should be part of deployment approval. Gartner recommends defining execution-success thresholds, explicit standards for acceptable task completion, alongside risk controls and the cost of meeting compliance requirements. “Refuse to greenlight deployments until scenarios are stress-tested against token-price swings and compliance expenses,” Sommer and Zimmerhansl advise. Approval therefore requires examining how changes in those conditions affect the workflow’s economics.

After deployment, the same success standards need to connect spending with business results. Gartner recommends including value per outcome in product planning and requiring each AI feature to forecast and track spending against a defined measure. Tasks automated and cases successfully closed are two examples. These measures let product and engineering teams assess what useful work the system completes for its operating cost.

The resulting spending and outcome data should also determine whether a workflow continues in its current form. Gartner advises using those records to identify workflows that need improvement or retirement. Improvement may mean changing model routing, revising execution or reconsidering the workflow’s role in the product. Retirement becomes a product decision when measured results no longer justify the expense.

Key executive takeaways

• Budget for total workflow costs: Gartner predicts token costs will fall 95% by 2030, while agentic inference spending rises more than fivefold over its two-year forecast period. Finance and product owners need spending forecasts that account for task volume, capability choices and execution frequency.
• Account for autonomous activity: Validation, retries and agent-to-agent communication add processing beyond visible user requests. Engineering teams need to track background activity and constrain execution to control spending.
• Match model capability to the task: Gartner’s modeling puts provider costs for planning and learning at eight to 10 times those of basic workflows. Platform teams can use model routing to reserve advanced capabilities for stages that require them.
• Measure the value of faster service: Gartner has observed customer success agents reducing response times by 99%, but profitability depends on total workflow costs. Product owners need to connect service improvements to measurable business outcomes.
• Make deployment approval an economic decision: Approval criteria need execution-success thresholds, risk controls and stress tests for token-price changes and compliance expenses. After launch, product owners need cost-per-outcome data to decide which workflows to improve or retire.