AI infrastructure is ready now the real work begins

AI infrastructure is no longer the primary barrier

For years, enterprise AI was limited by infrastructure. Only the largest technology companies could afford the computing power, data platforms, and engineering talent needed to build advanced AI systems. That is no longer true. Today, enterprise platforms such as Databricks have made AI infrastructure widely available. The technology itself has become accessible.

That changes the conversation. The competitive advantage is no longer about who owns the best infrastructure. It is about who can use it effectively.

A live poll during a BairesDev webinar made this shift very clear. Forty-six percent of attendees said their data platform is still used mainly for analytics and reporting. Not a single respondent selected machine learning and AI development as the platform’s primary purpose. The infrastructure exists, but many organizations are still operating with a mindset built for traditional business intelligence instead of AI-driven products.

This is an important distinction for executives. Buying AI technology is relatively straightforward. Building an organization that consistently delivers AI into production is much harder. That requires different engineering practices, stronger governance, cleaner data, and leadership that is willing to rethink how technology teams work.

The organizations that move first on these capabilities will have an advantage that is difficult to copy. Infrastructure can be purchased. Organizational capability has to be built over time.

Brett Berhoff, Founder and CEO of Strategy.xyz and BairesDev Fellow, guided the discussion around this shift. Together with Jody Mulkey, Chief Technology Officer at First American, JT Hwang, Chief Technology Officer at GoodLeap, and Ayman El-Ghazali, Senior Solutions Architect at Databricks, the panel consistently returned to the same conclusion: the next phase of enterprise AI is not about better models. It is about building the conditions that allow those models to create business value at scale.

For leadership teams, this changes investment priorities. Spending should move beyond infrastructure toward developing internal capabilities, modern engineering processes, governance frameworks, and cross-functional collaboration. These areas may receive less attention than new AI models, but they determine whether AI becomes a production capability or remains a series of isolated experiments.

Production AI requires platform-level support for probabilistic systems

Most enterprise software has been built around deterministic rules. Given the same input, the system produces the same output every time. AI does not work that way. Large language models generate probabilistic responses, which means there can be multiple valid outputs for the same request. That changes how software must be designed, tested, and operated.

This is where many organizations encounter their first major challenge. Their engineering teams are highly experienced, but their experience is rooted in traditional software development. Production AI introduces new requirements, including evaluating model quality, managing uncertainty, testing prompts, monitoring model behavior, and applying guardrails that keep outputs reliable and safe.

Jody Mulkey, Chief Technology Officer at First American, described how this became clear after her approximately 2,000-person technology organization began building AI-driven applications about two and a half years ago. Rather than attempting to train every engineer to become an AI specialist, the company built these capabilities directly into its internal platform.

Evaluation harnesses, experimentation frameworks, and governance controls became part of the development environment itself. Engineers could focus on solving business problems while the platform handled many of the AI-specific requirements behind the scenes. As Mulkey explained, the goal was to democratize the ability to build AI agents across the enterprise instead of relying on a small group of experts.

This approach has significant strategic value. AI adoption does not scale by creating more specialists. It scales by making AI development easier, safer, and more consistent for existing engineering teams. Organizations that depend on a handful of experts often struggle to expand successful pilots into enterprise-wide capabilities.

JT Hwang, Chief Technology Officer at GoodLeap, framed the shift from another perspective. Enterprise AI capabilities that were once available only to hyperscalers are now within reach of what he called “the other 99%.” Access to technology is no longer the limiting factor. The real challenge is whether organizations have built platforms and operating models that allow engineers to use those capabilities effectively.

For executives, this has direct implications for investment decisions. AI success depends less on hiring large numbers of specialized AI engineers and more on creating internal platforms that standardize best practices, reduce development risk, and make AI accessible across the organization. That approach improves consistency, accelerates delivery, and creates a foundation that can support AI initiatives long after the first projects reach production.

High-quality, consistently defined data is a prerequisite for reliable AI

AI systems only perform as well as the data they receive. That has always been true, but it becomes much more important when AI moves from assisting people to making recommendations or taking actions inside business processes. If the underlying data is inconsistent, incomplete, or poorly defined, the AI will produce results that appear confident but are not reliable.

Many organizations underestimate this challenge because humans naturally resolve ambiguity. AI does not. It works from the information and definitions it has been given. If different departments use the same business term in different ways, the AI has no built-in understanding of which interpretation is correct.

JT Hwang, Chief Technology Officer at GoodLeap, shared a practical example from his company’s operations managing home battery systems. A seemingly simple question about how many batteries the company has does not have a single answer. The result depends on whether someone means active batteries, inactive batteries, remotely manageable batteries, or batteries that can actually be controlled. A person would normally ask a follow-up question before responding. An AI agent may not.

To solve this problem, GoodLeap built a semantic layer beneath its AI systems. This creates one consistent definition for each business concept within every domain. The objective is straightforward: ensure that every AI application works from the same understanding of the business.

This investment often receives less attention than new AI applications, but it has a much greater impact over time. As organizations deploy more AI systems, consistent definitions reduce errors, improve trust, and make it easier to scale AI across multiple business functions.

Ayman El-Ghazali, Senior Solutions Architect at Databricks, said he sees similar issues across hundreds of customer environments. Many problems are not advanced AI challenges. They are basic data quality issues that already existed before AI was introduced. He pointed to examples such as organizations calculating averages from already averaged numbers, creating mathematically incorrect results before AI even enters the process. AI simply exposes these weaknesses more quickly because it can process large volumes of data without questioning whether the underlying information is correct.

Data quality and governance tied with demonstrating business value and ROI as the largest obstacles to building AI-powered applications, with each receiving 31% of responses in the live poll.

For executives, this should influence investment priorities. Companies often focus on selecting models or evaluating vendors, but the larger opportunity is building a disciplined data foundation. Clear business definitions, strong governance, and reliable data pipelines increase the value of every AI initiative that follows. Without that foundation, each new AI project inherits the same problems and requires additional effort to compensate for them.

Governance, compliance, and security must be integrated from the outset

Many AI projects begin with experimentation. That is useful for learning, but production systems operate under very different conditions. Once AI becomes part of customer interactions or business operations, governance, compliance, and security become core design requirements rather than implementation details.

One of the most common mistakes organizations make is treating governance as a final review before deployment. That approach creates unnecessary delays, higher costs, and greater operational risk. It is significantly more effective to define governance requirements before development begins so that they become part of the system architecture from the start.

Jody Mulkey, Chief Technology Officer at First American, explained that this was the company’s approach from the beginning. Because First American operates in a highly regulated industry and owns a bank regulated by the Federal Reserve Board, legal, compliance, and information security teams participated early in the AI development process. Rather than slowing development, this collaboration established clear requirements that engineering teams could build around from the beginning.

This approach is becoming increasingly important across industries. AI regulations continue to evolve globally, and customers expect organizations to demonstrate responsible use of AI, particularly when personal information or high-impact decisions are involved. Companies that build governance into their operating model are better positioned to adapt as regulatory expectations change.

JT Hwang, Chief Technology Officer at GoodLeap, reinforced this point from an engineering perspective. He noted that even before AI, complex technology projects often required disproportionate effort during the final stages of production deployment. His observation that “the last 10% is 90% effort” remains true for AI. Governance requirements become far more difficult to address when they are postponed until late in the development process.

For business leaders, governance should not be viewed only as risk management. It is also an accelerator for enterprise adoption. Clear policies, defined responsibilities, and established security controls give teams greater confidence to deploy AI across more business functions. They also reduce uncertainty for regulators, customers, employees, and boards of directors.

Organizations that consistently succeed with enterprise AI tend to treat governance as a capability that enables scale.

Effective production architecture simplifies governance and ensures security

Building an AI application is only the beginning. Operating it reliably over months and years requires architecture that keeps governance, security, and maintenance manageable. As AI agents become more involved in business processes, organizations need clear answers to practical questions: Who is responsible for an AI agent’s actions? What data can it access? How do you know it continues to perform correctly as models change?

The panel emphasized that these challenges should be addressed through architecture rather than manual oversight.

Jody Mulkey, Chief Technology Officer at First American, described one example. Instead of creating separate digital identities for AI agents, First American allows agents to inherit the permissions of the employee who invokes them. This approach avoids introducing another layer of identity management while ensuring that agents operate within the same access rights already assigned to users.

Mulkey acknowledged that independent AI identities may eventually become necessary for some organizations. However, her team’s priority has been to reduce unnecessary complexity while capturing the value AI can already deliver today. For many enterprises, that is a practical strategy. Organizations do not need to solve every future architectural challenge before they begin generating business value.

Security follows the same principle. JT Hwang, Chief Technology Officer at GoodLeap, explained that protecting personally identifiable information (PII) requires controls at the data layer itself. Prompt instructions can reduce certain risks, but they should not be treated as the primary security mechanism. If an AI system has unnecessary access to sensitive information, the organization has already increased its exposure.

For executives, this highlights an important distinction. AI governance is not simply about defining policies. It is about ensuring that those policies are enforced by the underlying technology. Access controls, identity management, auditability, and security should be built into the platform so they operate consistently across every AI application.

The discussion also emphasized the importance of continuous evaluation. Unlike traditional software, AI models evolve. Providers release new versions, existing models are retired, and organizations regularly adjust prompts, workflows, and business rules. Every change introduces the possibility of unexpected behavior.

Mulkey explained how this affected First American when an OpenAI model supporting one of its early AI products was deprecated. Because the company had already invested in comprehensive evaluation harnesses, the team was able to validate the new model against existing performance standards without rebuilding the entire application. The evaluation framework provided confidence that the system continued to meet business requirements after the migration.

For leadership teams, this demonstrates why evaluation should be treated as core infrastructure rather than an optional testing activity. Organizations that continuously measure AI performance can adopt new models more quickly, reduce operational risk, and avoid costly redevelopment when the AI ecosystem changes.

Redesigning workflows unlocks greater business value

Many organizations begin their AI journey by automating individual tasks. That approach can produce measurable improvements, but it rarely delivers the full value AI makes possible. The larger opportunity comes from redesigning entire business processes around what AI can now do.

Jody Mulkey, Chief Technology Officer at First American, emphasized that a person’s job consists of many connected activities rather than isolated tasks. Improving one step may increase efficiency, but it does not fundamentally change how work moves through the organization. Greater gains come from reconsidering the complete workflow.

She estimated that simply automating existing tasks typically produces productivity improvements of 20% to 40%. Those results are meaningful, but they represent only part of the opportunity. Organizations that redesign the workflow itself can achieve substantially larger improvements because they remove unnecessary handoffs, reduce delays, and allow AI to perform work that was previously limited by manual processes.

First American demonstrated this with its quality control operations. Previously, 15 employees reviewed client deliverables, covering approximately 37% of total volume while prioritizing the most demanding customers. After redesigning the process around AI, five employees now oversee 100% of the workload. The improvement was not only about reducing manual effort. AI also applies the same review criteria consistently across every case, improving coverage and reducing the variability that naturally occurs in human review.

Mulkey also observed the same pattern within software engineering. Teams that simply used AI coding assistants experienced incremental gains. The teams achieving significantly stronger results changed the entire development process. Engineers spent more time defining specifications, reviewing outputs, and orchestrating AI agents rather than writing every line of code themselves. This shifted their focus toward higher-value work while allowing AI to handle much of the implementation.

JT Hwang, Chief Technology Officer at GoodLeap, reinforced this broader perspective. He argued that many business processes were originally designed around human constraints. As AI removes some of those constraints, organizations should reassess how work is organized instead of assuming existing workflows remain the best approach.

For executives, this is one of the most important strategic decisions in enterprise AI. Measuring success only through task automation can lead organizations to underestimate AI’s potential. The companies that create lasting competitive advantage will redesign operating models, decision flows, and customer experiences to reflect AI’s capabilities. That requires changes in leadership, organizational design, and business processes.

Sustainable enterprise AI depends on building the operational conditions around the model

The discussion reached a clear conclusion. AI models are improving rapidly, enterprise infrastructure is widely available, and the technology is capable of supporting real business applications. The biggest challenge is no longer the model itself. It is everything an organization builds around it.

Many companies still concentrate their AI strategy on selecting models or experimenting with new tools. Those decisions matter, but they are unlikely to determine long-term success on their own. Sustainable AI depends on operational discipline. Organizations need trusted data, consistent governance, resilient architecture, continuous evaluation, and workflows designed to take advantage of AI rather than simply accommodate it.

These capabilities reinforce one another. High-quality data improves model performance. Governance reduces operational and regulatory risk. Strong architecture makes systems easier to scale and maintain. Continuous evaluation allows organizations to adopt new models without disrupting business operations. Workflow redesign ensures that AI creates measurable business outcomes instead of isolated productivity improvements.

The panelists consistently returned to this broader perspective. None of them identified model capability or infrastructure availability as the primary obstacle. Instead, they focused on organizational readiness and execution. This reflects the current state of enterprise AI. The technology has advanced faster than the operating models needed to support it.

Brett Berhoff, Founder and CEO of Strategy.xyz and BairesDev Fellow, moderated a discussion that repeatedly emphasized this point. Jody Mulkey, Chief Technology Officer at First American, JT Hwang, Chief Technology Officer at GoodLeap, and Ayman El-Ghazali, Senior Solutions Architect at Databricks, approached the topic from different industries and technical backgrounds, yet their recommendations were remarkably consistent. Success comes from building systems that can reliably support AI over time.

For executives, this changes how AI investments should be evaluated. Early demonstrations and pilot projects can generate enthusiasm, but they rarely reveal the operational requirements of production environments. Leadership teams should measure progress by asking different questions. Can AI applications be governed consistently? Is the underlying data trusted across the business? Can models be updated without disrupting operations? Are evaluation frameworks in place to verify quality over time? Can teams deploy AI repeatedly rather than treating every project as a one-off initiative?

Organizations that answer these questions positively are building an AI capability instead of a collection of AI projects. That distinction matters because competitive advantage increasingly comes from repeatability. The ability to deploy AI safely, efficiently, and consistently across multiple business functions will produce greater long-term value than isolated successes.

The next phase of enterprise AI will not be defined by access to technology. Most organizations can now obtain capable models and modern AI infrastructure. The leaders will be those that create the operational environment where those technologies can deliver measurable business results at scale, adapt as the technology evolves, and remain trusted by customers, regulators, employees, and shareholders.

The bottom line

Enterprise AI has entered a different phase. The conversation is no longer centered on whether the technology is capable enough. For most organizations, it is. The question is whether the business is prepared to use it consistently, responsibly, and at scale.

That requires a broader view of AI investment. Models will continue to improve, and infrastructure will become even more accessible. Those advances will benefit every organization. What will separate market leaders is everything that sits around the technology: trusted data, governance embedded from the start, platforms that simplify development, continuous evaluation, and workflows redesigned to take advantage of AI’s strengths.

For executives, this is ultimately a leadership challenge rather than a technology challenge. AI adoption cannot remain confined to innovation teams or isolated pilots. It requires alignment across technology, operations, legal, compliance, security, and the business itself. Organizations that build these capabilities into their operating model will be able to deploy AI repeatedly, adapt as the technology evolves, and generate value long after individual models have been replaced.

The companies that succeed over the next several years will not necessarily be the ones with access to the newest model. They will be the ones that build an organization capable of turning AI into a reliable business capability. That is where durable competitive advantage will come from, and where the greatest returns on AI investment are likely to be realized.