Your AI costs may start before inference

Some AI compute waste starts before the GPU

Some AI spending that looks like a compute problem begins in the data pipeline. When unnecessary information reaches a model, organizations use inference resources to process it.

The FinOps Foundation’s 2026 State of FinOps survey reports that 73% of enterprises say their AI costs have already outpaced what they budgeted. That figure shows budget pressure, but by itself does not identify poor data practices as the cause.

The executive question extends beyond GPU use and model pricing: what information is the organization paying those resources to process?

The warehouse habit that becomes expensive at inference

Data teams often collect broadly. A data warehouse can hold large datasets while queries select the records needed for a particular task.

AI systems create a different decision point. Raw logs, whole tables, redundant files, and irrelevant history can enter prompts or vector databases, which store representations of data so applications can retrieve relevant material for a model.

Inference makes the volume of transmitted context economically relevant when processing cost varies with input volume. “Context” is the information supplied to a model as input for a request. More context gives the model more input to process.

Broad retention can still support analytics, audit, and future uses. Sending data to an AI system creates a separate processing decision. Teams should decide which retained information deserves inference resources before it reaches the model.

Filtering upstream can reduce downstream work

Upstream filtering can remove irrelevant information before inference. Where processing cost varies with input volume, this can reduce unnecessary processing.

Freshness creates a related problem. Information can lose value between collection and use. Stale inputs may no longer fit the task.

Validation addresses another downstream risk. Consider a producer that changes a field name while several systems still expect the old schema, the defined structure and data types of a record. Those consumers can fail when they receive an unexpected structure.

A data contract is an agreement that defines the structure and expectations for data shared between systems. A pipeline can validate an event against that contract before admitting it to a shared stream. A schema registry, a service that records and manages data structures and their versions, can help systems manage changes over time.

These problems create different cost paths. Irrelevant records can add processing. Stale records can feed outdated information into a decision. Malformed records can cause downstream failures. Input control therefore affects both AI operating costs and data quality.

Agents raise the stakes for pipeline validation

An AI agent is a system that can use model outputs to choose or execute actions. When the system relies on incoming data, input errors can affect those actions.

A stale or malformed record can feed a decision based on inaccurate or outdated information. For systems authorized to act, the consequence depends on what actions the system can take.

Validation therefore has two roles: protecting pipeline reliability and controlling which inputs reach systems authorized to act. The level of control should match the consequence of a bad decision.

Make input efficiency part of AI FinOps

AI FinOps reviews can examine the inputs that create compute demand alongside the infrastructure supplying it. Teams can review filtering, freshness requirements, schema and data-contract enforcement, information assembled as model context, and failure handling across downstream systems.

Stream-time processing is one possible implementation. Apache Flink is a stream processing engine that can process data as it moves through a system.

The design principle is technology-neutral: decide what data is relevant and valid before it reaches expensive downstream processing.

Lakes, warehouses, and broad retention can remain part of the architecture. Teams can set explicit boundaries around information sent to prompts, retrieval systems, streams consumed by AI applications, and GPU clusters. Those boundaries place the compute-spending decision where data becomes model input.

Key highlights

• AI costs can start before the GPU: Leaders should examine what data reaches models, not just GPU utilization and model pricing, because unnecessary context can create avoidable inference demand.
• Treat inference differently from data retention: Keeping broad datasets can support analytics and audit needs, but not all retained information should become model input. Set explicit rules for what enters prompts and retrieval systems.
• Filter and validate data upstream: Remove irrelevant or stale inputs and enforce schemas and data contracts before expensive downstream processing to reduce unnecessary work and improve data quality.
• Apply stronger controls to agent inputs: Agents can turn bad data into actions, increasing the consequences of stale or malformed inputs. Match validation requirements to the potential impact of an incorrect decision.
• Make input efficiency part of AI FinOps: Review filtering, freshness, validation, context assembly, and failure handling alongside infrastructure costs. Put spending controls where data becomes model input.