Why multi model AI systems fail more often than enterprises expect

Enterprises overestimate the benefits of combining multiple AI models

Many enterprises assume that combining several AI models will naturally produce better results. The logic seems reasonable. One model is good at coding, another at reasoning, another at general knowledge. If each model has different strengths, then together they should cover each other’s weaknesses.

The latest research suggests this assumption is incomplete.

The real limit is not how often models make different mistakes. It is how often they all make the same mistake on the same prompt. The researchers call this the “co-failure ceiling.” Once every model in a pool reaches the wrong answer together, no router, voting system, or orchestration layer can recover the correct answer. That shared failure becomes the maximum possible accuracy for the entire system.

This changes how executives should think about AI investments. Many organizations are spending significant engineering effort building routing platforms because they expect model diversity to create meaningful gains. Those gains may never appear if the underlying models share the same blind spots.

The study examined 67 frontier models from 21 AI providers. Using traditional pairwise error correlation, the researchers predicted that every model would fail together on only 2.3% of questions in the MATH-500 benchmark. The actual co-failure rate was 5.2%. In other words, conventional metrics underestimated simultaneous failures by approximately 2.25 times.

That difference is important. It means many organizations are planning AI architectures based on optimistic assumptions rather than measurable limits.

Josef Chen, author of the research paper, argues that the real constraint is shared failure across models rather than model diversity itself. He explains that existing evaluation methods cannot detect a subset of difficult prompts where virtually every frontier model fails together. This hidden failure pattern creates a ceiling that orchestration alone cannot overcome.

For executives, the implication is straightforward. Before investing in additional models, ask a more fundamental question: are the models actually failing differently, or are they failing together? If the answer is the second, adding more models increases complexity without changing the outcome.

This also changes how AI performance should be measured. Traditional benchmarks often focus on the average accuracy of individual models. Enterprise deployments should also measure how frequently every candidate model fails on the same business-critical tasks. That provides a much better estimate of the maximum value orchestration can deliver.

The opportunity is still significant. AI models continue to improve rapidly, and shared failure rates may decrease over time. But decisions should be based on measurable evidence, not assumptions about diversity automatically producing better results.

Multi-model orchestration introduces significant operational costs that can outweigh its benefits

Every additional AI model brings more than another API call. It adds operational complexity across the entire technology stack.

Most enterprise orchestration strategies fall into three categories. Model routers decide which model should receive each request. Cascades begin with a cheaper model and only escalate to a more capable model when confidence is low. Mixture-of-Agents (MoA) systems ask several models the same question and combine their responses into a final answer.

Each approach has real costs.

Latency increases because requests move through more processing steps. Infrastructure becomes harder to maintain because multiple providers, APIs, and model versions must stay synchronized. Security and governance become more demanding as sensitive data may flow across several external platforms instead of one. Monitoring also becomes more difficult because failures can originate from the routing system, the models themselves, or the interaction between them.

These costs rarely appear in simple cost-per-token calculations, but they directly affect reliability, operational efficiency, and total cost of ownership.

Many organizations justify this additional complexity by measuring pairwise error correlation. If two models usually fail on different questions, the assumption is that combining them will improve overall performance.

The study challenges this assumption.

Low pairwise correlation does not necessarily produce higher system accuracy. If the models still fail together on the most difficult prompts, the expected benefit from orchestration largely disappears. Organizations may end up paying the operational cost while receiving only marginal improvements in production.

Josef Chen highlighted this issue in comments to VentureBeat. He noted that teams often accept higher latency, greater infrastructure complexity, and multi-provider operational overhead because they expect a “diversity dividend” later. According to the research, that dividend usually does not arrive because today’s leading AI models frequently agree, and they also fail on many of the same difficult questions.

For executives, this is a capital allocation question as much as a technical one.

Building orchestration platforms requires engineering resources, ongoing maintenance, vendor management, security reviews, compliance oversight, and continuous evaluation as new models are released. Those investments should only be made if measurable performance improvements justify the additional operational burden.

In many enterprise environments, investing those same resources in deploying a stronger frontier model, improving evaluation datasets, strengthening governance, or redesigning workflows to include verification may produce a better return.

The broader lesson is simple. Complexity should only be introduced when it creates measurable business value. AI architecture is no different. More models do not automatically produce better outcomes. In some cases, they simply produce a more expensive system.

Combining models of unequal quality can reduce accuracy instead of improving it

Many AI teams assume that adding more models will increase accuracy. That assumption only holds if the models contribute useful and comparable expertise. When there is a large gap in quality between models, the opposite can happen.

The study found that simple majority voting can make a system worse. If weaker models consistently outnumber a stronger one, they can override the correct answer. The result is lower overall accuracy, even though more models are involved in producing the response.

This is an important finding because majority voting is one of the simplest and most common approaches to multi-model systems. It is easy to implement and appears to offer greater reliability. The research shows that this confidence is often misplaced when model quality is uneven.

The researchers recommend combining only models that operate within a similar quality range. When that is not possible, they suggest choosing the strongest individual model instead of building an ensemble with weaker alternatives.

Their experiments support this conclusion. According to the study, naïve majority voting across models of unequal quality produced a negative average gain of approximately 10 percentage points on a challenging evaluation set. Rather than improving performance, the ensemble consistently reduced it.

Josef Chen, author of the paper, summarized the problem directly. He told VentureBeat that “naive majority voting across unequal models had negative mean gain (minus 10 points on our hard mix): diverse-but-weaker members outvote the strong one.” His recommendation was equally direct: organizations should “combine only models within a matched quality band.” If that is not possible, the better investment is usually the strongest single model available.

For executives, this has immediate procurement implications.

Many enterprises evaluate AI vendors independently, selecting the strongest model in each specialized category. That process does not guarantee those models will perform well together. An ensemble should be evaluated as a complete system rather than as a collection of individually strong products.

This also changes how organizations think about cost optimization. Replacing one premium model with several lower-cost models may reduce infrastructure spending on paper, but the savings can disappear if accuracy declines and human review increases. Lower-quality outputs often create additional operational costs through manual verification, customer dissatisfaction, or downstream process failures.

The better strategy is to evaluate total business performance rather than the price of individual models. In many situations, a single high-performing frontier model delivers more predictable results than a larger collection of mixed-quality alternatives.

Diverse model ensembles can outperform repeated use of the same model, but only under the right conditions

The study does identify one situation where model diversity creates measurable value.

Many organizations use a technique known as Self-MoA, or Self Mixture-of-Agents. Instead of consulting different AI models, the system queries the same premium model multiple times and combines the responses into a final answer. This approach introduces variation through repeated sampling while relying on a single model.

The researchers found that a carefully selected group of different models can outperform this approach, but only when two important conditions are met.

First, the models must have similar overall capability. Second, they must make different mistakes rather than repeating the same errors. When both conditions are satisfied, diversity becomes an advantage because each model contributes information that the others may not provide.

This is a more demanding requirement than simply selecting models from different providers or with different areas of specialization. True diversity comes from complementary strengths combined with comparable quality. Without both, the expected gains become much smaller.

It is also important to separate this finding from the broader conclusions of the paper. A stronger ensemble does not eliminate the co-failure ceiling discussed elsewhere in the research. Even a well-designed ensemble remains limited by prompts that cause every participating model to fail simultaneously.

Josef Chen emphasized that matched quality is the key requirement. According to the research, when quality is balanced, a diverse ensemble with low pairwise error correlation performs better than a high-correlation Self-MoA configuration. That finding supports selective use of ensembles rather than broad adoption across every workload.

For business leaders, the message is one of discipline rather than expansion.

An effective ensemble is not created by increasing the number of models. It requires careful evaluation, continuous testing, and evidence that each additional model contributes unique value. Every model added to production should justify its operational cost through measurable improvements in accuracy, reliability, or business outcomes.

This also suggests that AI architecture should remain flexible. As frontier models continue to improve, a diverse ensemble that performs well today may no longer offer meaningful advantages tomorrow if one model advances significantly ahead of the others. Regular evaluation should therefore become part of AI governance, ensuring that architecture decisions continue to reflect current model performance rather than historical assumptions.

For enterprises pursuing multi-model strategies, success depends less on model count and more on disciplined selection, ongoing measurement, and a clear understanding of where diversity creates genuine business value.

The “co-failure ceiling” is a fundamental limit on the accuracy of multi-model AI systems

The central contribution of the study is the introduction of the “co-failure ceiling” as a practical way to measure the limits of multi-model AI systems.

Most organizations evaluate models by looking at individual accuracy or pairwise error correlation. Those metrics are useful, but they do not answer the most important operational question: how often do all available models fail on the same prompt?

That shared failure rate defines the maximum accuracy any orchestration system can achieve. It does not matter how advanced the routing algorithm is or how many models participate. If every model produces an incorrect answer, the system has no path to a correct result.

This is especially relevant for enterprise workloads involving difficult reasoning, complex mathematics, or open-ended decision-making. These tasks often expose capability limits that are shared across today’s frontier models rather than isolated to a single provider.

The researchers demonstrated this gap using the MATH-500 benchmark, an open-ended mathematics evaluation. Standard statistical methods based on pairwise error correlation predicted that all models would fail together on only 2.3% of questions. The observed co-failure rate was 5.2%.

That difference is significant. Traditional evaluation methods underestimated simultaneous failures by approximately 2.25 times. For organizations making infrastructure decisions based on these estimates, the expected return from orchestration may be substantially overstated.

The study evaluated 67 frontier models from 21 providers, including GPT-5.5, Claude Opus 4.8, and Gemini 3.1 Pro. The consistency of the findings across such a broad set of leading models suggests that this is not an issue affecting only one vendor or architecture.

Josef Chen identified the source of this limitation as what the researchers call a “common-mode atom.” He explained to VentureBeat that this represents “a slice of queries on which the entire market fails together, which no pairwise statistic can see.” He added that “adding a 20th model to your pool doesn’t buy tail coverage. The tail is shared.”

This finding has strategic implications for executives.

Many AI roadmaps assume that adding more models will steadily reduce risk. The research suggests that this relationship eventually reaches a limit. Beyond that point, additional models increase operational complexity without materially improving reliability on the hardest problems.

Organizations should therefore distinguish between two separate questions. First, how accurate are individual models? Second, where do those models fail together? The second question is often more valuable when designing enterprise AI systems because it determines the practical ceiling of production performance.

This perspective also reinforces the importance of capability improvements at the foundation model level. Routing strategies and orchestration frameworks can optimize existing performance, but they cannot overcome shared capability limits that exist across the market.

Task design has a major impact on AI reliability, and structured tasks reduce shared failures

The study shows that the way work is presented to an AI model can significantly influence overall system reliability.

Open-ended generation creates more opportunities for models to produce different incorrect answers or to fail together. When there is no objective way to verify correctness, routing systems have little information to determine which response should be trusted.

The researchers found that changing the format of a task can dramatically alter co-failure rates. When graduate-level science questions from the GPQA benchmark were converted from multiple-choice to free-response, the simultaneous failure rate increased to 12.7%.

This result highlights that evaluation is not only about model capability. It is also about task design.

Josef Chen argued that organizations should convert generation problems into verification problems whenever possible. In his words, “Anywhere you can convert generation into verification or constrained selection (structured outputs, checkable answers, execution tests), you reopen the ceiling.”

This recommendation has broad implications for enterprise AI deployments.

Many business processes already have objective definitions of success. SQL queries either execute correctly or they do not. JSON outputs either match the required schema or they fail validation. Financial calculations can be checked automatically. Software code can be tested. Data extraction can be compared against known values.

By designing workflows around these measurable outcomes, organizations can improve reliability without waiting for the next generation of foundation models. Automated validation also reduces the amount of manual review required, allowing teams to identify errors before they reach customers or downstream systems.

Executives should view workflow design as an important part of AI strategy rather than focusing exclusively on model selection. Two organizations using the same frontier model may achieve very different business results depending on how they structure prompts, define outputs, and validate responses.

This finding also supports a broader governance principle. AI systems should be evaluated within the context of the business process they support, not as isolated technologies. A model that performs well in an unconstrained benchmark may produce inconsistent results in production if the surrounding workflow lacks verification. Conversely, a carefully designed process with structured outputs and automated checks can substantially improve reliability, even when using the same underlying model.

The practical opportunity is clear. Organizations cannot always control how quickly frontier models improve, but they can control how AI is integrated into business operations. Well-designed workflows, objective validation, and structured outputs remain some of the most effective ways to improve enterprise AI performance today.

Enterprises can estimate the value of multi-model orchestration before deployment using a simple statistical test

One of the most practical findings from the study is that organizations do not need to build a multi-model system before determining whether it is worth the investment.

The researchers recommend using the Clopper-Pearson bound, a well-established statistical method, to estimate the highest achievable accuracy of a model pool based on its observed co-failure rate. Rather than predicting average performance, this method provides a conservative estimate of the system’s practical limits when only a limited evaluation dataset is available.

This allows engineering teams to answer a critical business question early in the development process: is the expected improvement large enough to justify the additional cost and operational complexity?

The process is relatively straightforward. Organizations first create a representative evaluation dataset using real business tasks. Each candidate model is then tested on the same set of prompts, and the team records where every model succeeds and where every model fails together. Using those results, the Clopper-Pearson calculation estimates the upper bound of system performance without requiring the organization to build routing logic or conduct expensive production experiments.

Suppose a team evaluates five AI models on 50 representative queries and finds that all five models fail together on only two questions. At first glance, this appears to suggest a system capable of achieving approximately 96% accuracy. The Clopper-Pearson bound produces a more conservative assessment, indicating that because the evaluation sample is relatively small, the true co-failure rate could still be as high as 12%.

This distinction matters because executives often make investment decisions using optimistic projections derived from limited testing. Statistical confidence becomes especially important when AI systems support business-critical operations where even small error rates carry significant financial or regulatory consequences.

The researchers also emphasize that implementing this evaluation requires very little additional engineering effort.

Josef Chen explained that integration is “trivial: it’s a counting job over eval logs teams already produce, so it runs in the same CI stage as the eval suite and re-triggers whenever the model pool or the workload changes.”

For organizations that already maintain AI evaluation pipelines, this means the measurement can become part of standard governance rather than a separate project.

This approach also encourages continuous decision-making instead of one-time assessments. As new frontier models become available, enterprises can repeat the same evaluation using updated workloads and compare whether shared failure rates are improving. This creates an evidence-based process for deciding when to expand, replace, or simplify AI architectures.

For executives, the broader lesson extends beyond this specific statistical method. Major infrastructure investments should be supported by measurable evidence before resources are committed. AI orchestration should be treated as a business investment with defined performance thresholds rather than an assumption that more models will automatically create more value.

For many enterprise tasks, the strongest single frontier model delivers greater value than a complex multi-model architecture

The study reaches a conclusion that challenges a common assumption in enterprise AI strategy.

For tasks with objective, verifiable answers, organizations often achieve better results by deploying the strongest available frontier model rather than coordinating multiple lower-cost models through routing systems.

This finding applies particularly to workloads where correctness can be measured precisely. Examples include generating SQL queries that must execute successfully, extracting specific values from business documents, producing structured JSON that conforms to strict schemas, or completing other tasks where outputs can be automatically verified.

In these situations, additional orchestration frequently introduces more complexity than value.

A multi-model architecture increases the number of systems that must be managed, monitored, secured, and evaluated. If the routing layer cannot reliably identify which model will produce the correct answer for each request, the operational overhead remains while the expected performance gains become minimal.

The researchers conclude that meaningful improvements over the best single model generally require an exceptionally strong query-level routing signal. In practice, such routing signals are difficult to develop because the hardest questions often provide little indication of which frontier model, if any, will produce the correct response.

This is particularly relevant for organizations evaluating AI primarily through cost comparisons.

Choosing several lower-cost models instead of one premium model may appear financially attractive during procurement. However, the total cost of ownership includes engineering effort, governance, maintenance, monitoring, vendor management, testing, and the business impact of incorrect outputs. If orchestration produces only marginal accuracy gains, those indirect costs can outweigh any savings on inference.

The study also draws an important boundary around its conclusions. The researchers evaluated objectively verifiable benchmarks rather than subjective creative tasks. As a result, they note that it remains an open question whether these findings apply equally to areas such as marketing content, creative writing, or other work where multiple acceptable answers may exist.

This distinction is important for executive decision-making. AI strategy should not assume that one architecture fits every workload. Instead, organizations should classify use cases according to whether outputs can be objectively validated. That classification should influence both technology selection and investment priorities.

Josef Chen reinforced this long-term perspective when discussing how organizations should evaluate future AI systems. He noted that “the measurement costs nothing, so any team can track its own co-failure rate across model generations and watch whether the tail is closing.” He concluded that “the lever buyers hold is failure-mode heterogeneity and market churn, not model count.”

For business leaders, the message is clear. Success with enterprise AI is determined less by the number of models deployed and more by disciplined evaluation, measurable reliability, and alignment between the chosen architecture and the business problem being solved. As frontier models continue to improve, organizations that rely on evidence rather than assumptions will be better positioned to capture value while avoiding unnecessary complexity.

Final thoughts

Enterprise AI is entering a more disciplined phase. The conversation is shifting away from how many models an organization can deploy and toward how much measurable value those models actually create.

This research challenges one of the industry’s most common assumptions. Model diversity alone is not a reliable strategy for improving performance. If multiple models share the same failure modes, adding more of them increases operational complexity without changing the outcome where it matters most.

For executives, this reinforces an important principle. AI investments should be evaluated with the same level of rigor as any other technology investment. New infrastructure, additional vendors, and more sophisticated architectures should only be adopted when they deliver measurable improvements in business outcomes.

The encouraging news is that organizations have practical ways to make better decisions today. Measuring co-failure rates, building representative evaluation datasets, and validating AI against real business workflows provide a much clearer picture of expected performance than assumptions based on model diversity alone. These are relatively low-cost practices that can prevent costly architectural decisions later.

The research also highlights another opportunity. Many improvements in enterprise AI will come not only from stronger foundation models, but from better system design. Structured workflows, objective validation, and continuous evaluation can often improve reliability without increasing architectural complexity.

At the same time, this should not be interpreted as an argument against multi-model AI altogether. There will continue to be scenarios where carefully selected ensembles create meaningful value, particularly when models have comparable capability and genuinely complementary strengths. The key is proving that value through evidence rather than assuming it exists.

As frontier AI models continue to evolve, today’s conclusions will also evolve. Organizations that continuously measure performance, reassess their architectures, and adapt as capabilities improve will be in the strongest position to capture value. Those that rely on assumptions or industry trends alone risk building increasingly complex systems that offer diminishing returns.

The next competitive advantage in enterprise AI is unlikely to come from deploying more models. It will come from understanding exactly where AI succeeds, where it fails, and making investment decisions based on that evidence.