Rising token costs are forcing leaders to rethink AI

Rising AI costs are forcing companies to rethink whether to buy or build

The first phase of enterprise AI was largely about moving fast. Companies experimented with models, purchased software, expanded computing capacity, and trained employees. Now the economics matter much more. AI spending is accumulating across compute, software, and training, pushing executives to ask a basic question: should we continue buying AI products, or build more capabilities ourselves?

EY data shows why this question is becoming important. Three-quarters of senior leaders with ongoing AI investments say off-the-shelf software does not align with their IT needs. Nearly nine in ten businesses have already deployed or are piloting programs to develop AI internally. That is a substantial shift toward greater control over AI technology.

Building internally can give companies more control over integration, data, workflows, and product design. It can also reduce dependence on a single software provider. But building is not automatically cheaper. Internal AI requires engineering talent, infrastructure, security, model evaluation, maintenance, and continuous updates. Those costs can become significant, especially as the underlying technology changes quickly.

The right decision therefore goes beyond comparing a vendor subscription with the cost of an internal development team. Executives should evaluate total cost of ownership, including integration, compute consumption, governance, maintenance, employee time, and the cost of switching technologies later. They should also consider strategic value. If an AI capability is central to a company’s competitive advantage, greater internal ownership may make sense. For standardized functions, commercial products can remain faster and more economical.

The broader opportunity is to become more selective. Companies do not need to choose “buy” or “build” for everything. Many will use a hybrid model: purchase standard capabilities where differentiation is limited and invest internally where proprietary data, workflows, or intellectual property can create measurable business value.

AI vendors are responding as enterprises become more cost-conscious

Cost pressure is also changing the supplier side of the AI market. Enterprise customers increasingly want lower costs, greater transparency, and better control over consumption. AI providers are responding by changing prices and introducing tools designed to help customers manage spending.

OpenAI reduced the cost of its Luna and Terra models, while Oracle and AWS introduced product changes intended to improve cost management. One important caveat is required here: the supplied source names “Luna and Terra” as OpenAI models, but those model names should be independently verified before publication. They do not correspond to established OpenAI model names in the information provided here, so repeating the claim as independently confirmed fact would be inappropriate.

The underlying trend is nevertheless important for executives. Generative AI often has variable operating costs. Increased employee adoption, larger workloads, more complex requests, and higher inference volumes can increase spending even when the unit price falls. A cheaper model therefore does not necessarily produce a smaller total AI bill.

This changes how companies should evaluate vendors. Model capability remains important, but price per unit is only part of the equation. Executives should also examine how efficiently a system completes a task, how much usage can be measured and controlled, whether workloads can move between providers, and how pricing could change as deployments scale.

Competition among providers should continue to create opportunities for enterprise buyers. Falling unit costs can make new use cases economically viable, while improved cost controls can make budgets more predictable. Companies can use that competition to negotiate better terms and avoid unnecessary dependence on one platform.

The key is to connect procurement with measurable outcomes. A lower AI price is useful, but the objective is not simply to buy cheaper tokens or computing capacity. It is to produce more business value for each dollar invested. For C-suite leaders, that is the metric that ultimately matters.

Fast AI development makes long-term planning harder and financial discipline more important

AI changes quickly enough that a strategy approved today can face different technology, pricing, and competitive conditions within months. This creates a practical problem for executives: companies need to invest for the future without committing too much capital to assumptions that may become outdated.

Diasio captures this issue directly: “This is a technology that moves the goal posts every six weeks, so it’s understandable that executives haven’t figured out the roadmap or the right narrative for change.” Diasio adds that organizations need to choose initiatives intentionally and drive them toward financial value, or they risk “running in place.”

The implication is not that companies should wait for AI technology to stabilize. Waiting carries its own competitive risk. A stronger approach is to make investments in stages, with clear business objectives and defined criteria for continuing, expanding, redesigning, or ending a project.

That requires better measurement. Executives should connect AI programs to outcomes such as revenue growth, operating-cost reduction, employee productivity, customer retention, cycle-time improvements, or risk reduction. The appropriate metric depends on the use case. A customer-service system, for example, should be evaluated against service quality and economics rather than adoption numbers alone.

Leaders should also distinguish experimentation from scaled deployment. Experiments can test whether a technology works and whether employees will use it. Production investments face a higher standard: they need reliable performance, security, governance, integration, and a credible economic case. Separating these stages can prevent promising demonstrations from automatically becoming expensive long-term commitments.

The executive objective is therefore flexibility with accountability. Companies can move quickly while reviewing assumptions frequently. AI roadmaps should support changes in models and vendors without requiring the entire strategy to be rebuilt. The technology will continue evolving; investment discipline needs to evolve with it.

Poor visibility into AI usage is becoming a direct financial problem

Companies cannot effectively control AI spending if they do not know where the technology is being used, by whom, and at what cost. For many enterprises, that visibility remains surprisingly limited.

Flexera data released in June found that more than two-thirds of companies lack accurate visibility into AI software usage. Roughly three in five also reported a year-over-year increase in AI overspending. Together, these findings suggest that AI cost management is becoming a material operational issue rather than simply a procurement concern.

The challenge becomes more significant as adoption spreads across an organization. Different teams can purchase AI applications independently, employees can subscribe to overlapping services, and software platforms may introduce AI features with separate consumption charges. At the same time, API and model costs can fluctuate with actual usage. Traditional annual software budgets may not capture these patterns well.

More visibility does not mean restricting useful AI adoption. The goal is to understand consumption well enough to make better decisions. Executives need a consolidated view of which tools are being used, their owners, their costs, the business functions they support, and whether those applications produce measurable value.

This information also supports stronger governance. Companies can identify redundant subscriptions, unexpectedly expensive workloads, unused licenses, and AI services operating outside approved security or data policies. Finance, IT, procurement, security, and business teams therefore need a shared view rather than separate records of AI activity.

Cost controls should also account for value. A rapidly growing AI expense can be justified if it creates a larger improvement in revenue, productivity, or customer outcomes. Conversely, inexpensive software can still be wasteful if employees rarely use it or it produces little measurable benefit.

For C-suite teams, the objective is straightforward: make AI spending visible enough to manage. As deployment expands, knowing the total bill is no longer sufficient. Leaders need to understand what generates that bill and what the organization receives in return.

FinOps is expanding into AI as companies demand tighter control over variable technology costs

FinOps is becoming increasingly relevant to enterprise AI. The practice began as a way for finance, technology, and business teams to jointly manage cloud spending. AI introduces a new set of variable costs, including model usage, computing resources, software subscriptions, and consumption-based charges. As adoption grows, those costs need active management.

This matters because AI spending can change quickly with usage. Generative AI services may charge according to tokens, the units used to process inputs and generate outputs, or according to computing resources and other consumption measures. More users, longer prompts, larger workloads, or higher model usage can therefore increase costs rapidly. A budget based mainly on licenses or initial deployment expenses may provide an incomplete picture.

Applying FinOps principles can give executives better visibility. Companies can track AI consumption by product, department, application, model, or business process and then connect that spending with outcomes. This makes it easier to identify inefficient workloads, duplicate services, unexpectedly expensive applications, and opportunities to use lower-cost models when premium capabilities are unnecessary.

The Tokenomics Foundation reflects growing interest in formalizing this discipline. The organization was unveiled the previous month as an offshoot of the Linux Foundation and aims to help businesses bring enterprise-wide AI spending under control. The development suggests that AI cost management is becoming an organizational capability rather than an isolated finance or IT task.

For executives, governance should extend beyond reducing the monthly bill. The more useful question is whether each category of AI spending generates sufficient value. A workload with high token consumption may still make economic sense if it materially increases revenue, productivity, or service quality. A low-cost application that generates little value may deserve elimination.

Companies should therefore define ownership early. Finance can establish economic targets, technology teams can measure consumption and architecture efficiency, procurement can manage vendor terms, and business leaders can remain accountable for outcomes. Shared metrics help these groups make decisions using the same cost and performance information.

The objective is not simply cheaper AI. It is controlled, measurable AI economics. Companies that understand what they spend, why they spend it, and what they receive in return will be better positioned to scale successful applications while shutting down those that do not justify continued investment.

Key highlights

• Rethink the buy-versus-build decision: Rising compute, software, and training costs are changing AI economics. Leaders should compare total ownership costs and build internally where proprietary capabilities create meaningful differentiation.
• Push vendors harder on cost and value: AI providers are responding to cost pressure with pricing and cost-management changes. Use stronger competition to negotiate favorable terms while measuring total consumption and business outcomes.
• Keep AI investment plans flexible: Rapid advances can make long-term assumptions obsolete quickly. Fund initiatives in stages, establish clear financial and operational KPIs, and scale only when results justify further investment.
• Make AI spending visible: More than two-thirds of companies lack accurate visibility into AI software usage, according to Flexera, while roughly three in five report increased year-over-year AI overspending. Track usage, ownership, cost, and business value across the enterprise.
• Extend FinOps discipline to AI: Variable model, token, and compute costs require continuous financial management. Give finance, IT, procurement, and business teams shared accountability for connecting AI consumption with measurable returns.