What AI benchmarks get wrong about real world performance

Traditional AI benchmarks overlook real-world performance constraints

Most AI infrastructure discussions still focus on the obvious metrics: GPU count, training speed, cloud capacity, and benchmark scores. Those numbers matter, but they are only part of the system. The real question is much simpler. Can data consistently reach your compute resources when the environment becomes unpredictable?

That is where many AI deployments begin to struggle. Benchmark tests are usually performed in carefully controlled environments where networks behave exactly as expected. Production environments do not. Traffic patterns change every minute. Network latency increases unexpectedly. Individual storage nodes become slower or temporarily unavailable. These conditions are normal in large-scale enterprise systems, yet they are often absent from benchmark testing.

The result is a growing gap between expected performance and actual business outcomes. A system that appears highly efficient during testing may deliver far lower performance after deployment. GPUs that cost millions of dollars can spend valuable time waiting for data instead of processing it. When this happens, infrastructure that appears fully provisioned still fails to deliver its expected return.

Hunter Smit, Senior Manager of Product Marketing at F5, summarized the issue clearly: “Provisioning solves for capacity but not for delivery.” His point is that buying enough GPUs and storage does not automatically create an efficient AI platform. AI traffic is fundamentally different from traditional enterprise workloads. It is highly concurrent, arrives in bursts, and often requests data in random patterns that conventional storage networking was never designed to handle efficiently.

For executives, this changes how AI infrastructure should be evaluated. The conversation should move beyond hardware specifications and benchmark rankings toward end-to-end system behavior. Every layer between storage and compute contributes to business performance. If that path becomes unreliable, the most powerful AI hardware cannot compensate.

There is also a broader strategic implication. As organizations move AI into customer-facing products and core business operations, predictable performance becomes more valuable than peak performance. Customers experience the application as a complete service. They do not separate compute, storage, networking, or security into individual components. They simply notice whether the system responds quickly and consistently.

That is why more enterprises are treating data delivery as a strategic capability instead of a background infrastructure function. Technologies such as application delivery controllers (ADCs) and application delivery and security platforms (ADSPs) are increasingly being positioned between storage and compute to monitor system health, improve resilience, and maintain performance when network conditions inevitably change. The objective is AI that performs reliably in the environments where businesses actually operate.

Benchmark methodologies often produce overly optimistic performance outcomes

Benchmarks are useful, but only if they measure conditions that actually exist. Many AI infrastructure benchmarks do not. They are designed to produce the highest possible performance numbers. That creates a problem because enterprise AI systems rarely operate under ideal conditions.

One of the biggest factors is network latency. Latency is simply the delay between requesting data and receiving it. Even small increases in latency can significantly reduce the speed at which AI systems receive data from object storage platforms such as Amazon S3-compatible storage. Unlike traditional business applications, AI workloads often depend on thousands of GPUs working together simultaneously. Small delays become much larger when multiplied across an entire cluster.

Paul Pindell, Principal Solutions Architect for Technology Alliances at F5, explained that “meaningful testing has to introduce consistent latency into the path.” Without doing that, benchmark results represent an environment that production systems will almost never experience.

To examine this more closely, F5 and MinIO conducted throughput testing under degraded network conditions. Their results showed that S3 throughput declined rapidly as latency increased. Even relatively modest latency caused measurable performance loss. As latency approached the levels commonly found across long-distance network connections, throughput degradation became severe.

One result was particularly notable. The testing found that latency had a much greater impact on throughput than network jitter. That challenged the team’s initial expectations and provides an important lesson for infrastructure planning. Organizations often focus on reducing variability in network performance, but consistently lowering latency may have a much larger effect on AI performance.

For executives making infrastructure investment decisions, this changes the definition of successful testing. A benchmark should not answer whether a system performs well under perfect conditions. It should answer whether the system continues to perform when conditions become imperfect, because that is the environment customers and employees will experience every day.

This also has financial implications. Decisions based solely on benchmark results can lead organizations to invest heavily in additional compute resources while overlooking the actual bottleneck. If the data path cannot consistently supply information to GPUs, adding more GPUs simply increases idle capacity instead of increasing business value.

The more mature approach is to treat production conditions as part of the testing process from the beginning. Introducing realistic latency, degraded network paths, and storage node failures during evaluation produces infrastructure decisions that are far more likely to deliver predictable performance after deployment. In enterprise AI, consistency under real operating conditions is ultimately more valuable than impressive numbers generated in a controlled laboratory.

The efficiency of the storage-to-compute data path is critical to overall AI system performance and ROI

The discussion around AI infrastructure often starts with GPUs because they are expensive, highly visible, and easy to measure. That focus is understandable, but it is incomplete. A GPU only creates value when it has a continuous supply of data. If data arrives slowly or inconsistently, the GPU waits. That waiting time directly reduces the return on a company’s AI investment.

The storage-to-compute path includes much more than storage itself. Data moves through networking, databases, security controls, orchestration software, and distributed infrastructure before it reaches the compute layer. Every component contributes to overall system performance. If one part becomes a bottleneck, the entire AI pipeline is affected.

Tanu Mutreja, Senior Director of Product Management at F5, emphasized this point by noting that “in production environments, GPUs generate only as much value as the data path that feeds them.” This reflects an important shift in enterprise AI strategy. Organizations should evaluate infrastructure as an integrated system rather than as a collection of independent technologies.

The impact of an inefficient data path extends beyond lower GPU utilization. Training jobs can take longer to complete, delaying product development and experimentation. Inference systems may respond more slowly, affecting customer experience and business operations. Poor data delivery can also reduce the quality of AI outputs if models are unable to access relevant data efficiently when needed.

Operational costs increase as well. Organizations may replicate data unnecessarily across multiple locations to compensate for unreliable delivery, creating higher storage and network egress expenses. Engineering teams also spend more time diagnosing infrastructure issues instead of improving AI applications. As deployments grow, these operational inefficiencies become increasingly difficult and expensive to manage.

AI workloads are particularly sensitive to these problems because they typically run across large clusters of GPUs working simultaneously. Traditional enterprise applications often reduce the impact of storage delays through caching and buffering. Many AI workloads have fewer opportunities to hide infrastructure delays. Even relatively small increases in latency or temporary bandwidth constraints can affect large portions of a distributed training or inference workload at the same time.

For executives, this means infrastructure investment decisions should be evaluated against business outcomes. Higher GPU utilization, faster model delivery, improved application responsiveness, and lower operational complexity all contribute directly to stronger returns on AI investments. The organizations that optimize the complete data path will generally achieve greater value from the compute resources they already own before purchasing additional capacity.

Embedding intelligence into the AI data path is essential for adapting to production challenges

Enterprise architecture is changing because AI changes the way data is used. In many traditional systems, data was stored first and processed later. AI applications require continuous movement of data between storage and compute, often at very large scale and with demanding performance requirements. That makes the data path an active part of the application rather than a passive connection.

Tanu Mutreja explained that competitive advantage now depends on the amount of data an organization possesses, and on its relevance, lineage, security, and the ability to deliver it efficiently. This reflects a broader trend across the technology industry, where intelligence is increasingly being built directly into infrastructure instead of being added after the fact.

One practical example described is F5’s integration with MinIO. Within the F5 Application Delivery and Security Platform (ADSP), BIG-IP operates directly in the data path between storage and compute. Rather than simply forwarding requests, it continuously monitors the health of MinIO’s distributed storage nodes and makes routing decisions based on current operating conditions.

This capability becomes particularly valuable when storage nodes begin to degrade, which is expected over time in distributed environments. Without intelligent traffic management, requests may repeatedly reach unhealthy nodes before eventually finding one that can respond correctly. Those repeated retries increase latency, consume additional network resources, and reduce overall system efficiency.

Paul Pindell, Principal Solutions Architect for Technology Alliances at F5, explained that BIG-IP directs S3 client traffic toward healthy or less busy storage nodes so requests are processed as efficiently as possible. This allows the system to respond dynamically as infrastructure conditions change instead of relying on static routing decisions.

For business leaders, this approach has strategic value beyond technical performance. Infrastructure that continuously adapts to changing conditions supports more predictable application behavior, improves service availability, and reduces operational disruption. As AI becomes embedded in customer-facing products and critical business processes, these characteristics become essential rather than optional.

Embedding intelligence into the data path also strengthens long-term scalability. AI environments continue to grow in size and complexity, spanning more users, more data, and more infrastructure locations. Static architectures become increasingly difficult to operate under those conditions. Intelligent infrastructure that observes system health and adjusts automatically provides a more resilient foundation for future expansion while reducing the operational burden placed on engineering teams.

Multi-cloud and distributed AI deployments require centralized governance to manage digital sovereignty and resilience

As AI deployments expand, infrastructure decisions become about more than performance. Organizations are increasingly operating AI workloads across multiple public clouds, private data centers, and edge locations. This provides flexibility, but it also introduces new challenges around governance, compliance, and operational control.

One of the most important considerations is digital sovereignty. Regulations in many countries determine where data can be stored, how it can be processed, and who is permitted to access it. These requirements are becoming a core part of AI infrastructure planning rather than a compliance review that happens after deployment.

Hunter Smit, Senior Manager of Product Marketing at F5, explained that once AI pipelines span multiple regions and cloud providers, the conversation shifts from performance to control. Different jurisdictions impose different legal and regulatory requirements, meaning enterprise architecture must be designed to enforce those policies consistently across every environment.

This is contributing to a broader trend in enterprise AI. Some organizations are moving selected AI workloads from public cloud platforms back to infrastructure they own or directly govern. The objective is to reduce costs and also to improve visibility, strengthen governance, and maintain greater control over sensitive data and AI operations.

A centralized control layer helps address these challenges. Instead of tightly coupling applications to a single storage platform or cloud provider, organizations can introduce a management layer that applies consistent policies regardless of where data resides. This enables infrastructure teams to manage security, access controls, routing policies, and compliance requirements from a unified operational perspective.

For executives, this approach creates greater flexibility over time. Business priorities, regulatory requirements, and geographic expansion will continue to evolve. Infrastructure designed around centralized governance allows organizations to adapt without rebuilding AI applications every time data needs to move between environments or new compliance obligations emerge.

It also reduces operational complexity. Managing multiple cloud providers, storage platforms, and regional regulations independently creates fragmented operations and increases the likelihood of inconsistent policy enforcement. A unified governance model improves consistency while helping organizations balance resilience, compliance, and cost as part of a single enterprise strategy.

Managing the storage-to-compute connection as a controllable, engineered interface

Many AI architectures still assume that the connection between storage and compute is simply a transport layer. This assumption no longer reflects the realities of production AI. As workloads become larger and more distributed, the path itself requires active management.

Hunter Smit recommends treating the storage-to-compute connection as a managed control point rather than a direct connection. Introducing a full-proxy application delivery controller (ADC) between storage and compute provides continuous visibility into system health while allowing organizations to manage traffic intelligently as conditions change.

This creates several operational advantages. Health-based routing can automatically direct requests away from degraded storage nodes. Quality-of-service policies can prioritize critical workloads during periods of heavy demand. Security controls can be enforced directly within the data path instead of being added as separate layers elsewhere in the infrastructure. Together, these capabilities improve reliability without requiring changes to AI applications themselves.

The value of this approach increases as AI infrastructure grows. Distributed environments inevitably experience hardware failures, temporary network congestion, and changing workload demands. An engineered control point enables organizations to respond automatically to these events, reducing service disruption and maintaining more consistent application performance.

SecureIQLab independently validated F5 BIG-IP in storage deployments, confirming that this architecture can improve resilience while maintaining throughput. That is an important consideration for executives evaluating infrastructure investments, because resilience often comes with concerns about additional overhead or reduced performance. The validation suggests that organizations do not necessarily need to trade performance for greater operational stability.

From a business perspective, this approach shifts data delivery from an assumed capability to one that is actively managed and measured. That distinction becomes increasingly important as AI moves from experimentation into mission-critical operations. Reliable data delivery supports higher GPU utilization, more predictable application performance, stronger security enforcement, and more efficient use of infrastructure investments.

The broader lesson is that AI infrastructure should be evaluated as a complete operating system for data movement, not simply as a collection of compute and storage resources. Organizations that build visibility, control, and resilience directly into the storage-to-compute path will be better positioned to scale AI initiatives while maintaining operational consistency and maximizing long-term return on investment.

Main highlights

• Look beyond benchmark scores: AI benchmarks measure ideal conditions. Leaders should evaluate how infrastructure performs under latency, node failures, and unpredictable traffic because those factors ultimately determine business outcomes.
• Test for production: Infrastructure decisions based on best-case benchmarks can create costly performance gaps after deployment. Incorporate realistic network latency and degraded conditions into testing to better predict AI performance at scale.
• Optimize the full data path: GPUs deliver value only when they receive data consistently and efficiently. Leaders should invest in the entire storage-to-compute path to improve utilization, reduce operational costs, and maximize AI ROI.
• Build intelligence into infrastructure: AI platforms should actively monitor system health and adapt to changing conditions instead of relying on static routing. Intelligent data delivery improves resilience, application responsiveness, and operational efficiency without requiring changes to AI workloads.
• Make governance part of AI architecture: As AI expands across clouds and regions, centralized control becomes essential for meeting compliance, digital sovereignty, and security requirements. Design infrastructure that enforces consistent policies across all environments while maintaining operational flexibility.
• Treat data delivery as a strategic capability: The connection between storage and compute should be actively managed with visibility, health-based routing, security, and quality-of-service controls. This approach improves resilience, keeps expensive compute resources productive, and supports long-term AI scalability.