How AI workloads should shape your legacy modernization plans

AI adds another requirement to modernization

Expanding AI use puts pressure on legacy systems that may already be expensive to maintain and slow to change. AI needs substantial computing capacity, broad data access, continued IT spending and clear governance plans. Agentic systems, AI systems that take actions to complete tasks, bring those requirements into the operations they perform. So for chief information officers (CIOs), an AI investment decision also means deciding which underlying systems must change.

These infrastructure demands add to the concerns already driving modernization: cybersecurity exposure, slow software releases and high maintenance costs. IT leaders are committing large portions of their budgets to AI overhauls, but increasingly sophisticated models still carry security risks; AI agents have also breached government sites and other AI institutions. CIOs must account for those risks as they prepare systems for broader AI use. They need a clear view of what their intended deployments require.

Assess readiness for the intended AI workload

These requirements become especially important when AI agents perform work essential to an organization’s operations. In a July report, Google found that only 17% of IT leaders were confident their technology stack, the software and infrastructure supporting their operations, could support these mission-critical agents. Google has a commercial interest in wider AI adoption, which gives it a stake in how enterprises assess their readiness. Its finding measures leaders’ confidence in supporting this demanding work.

Alongside that confidence measure, Google’s report describes enterprises at several stages of modernization. Nearly half reported that they had begun updating legacy systems, while about one-quarter said they had started but considered themselves behind. Another 1% had yet to begin any modernization plans, and only 15% of tech leaders described their modernization as nearly or fully finished. These descriptions capture progress and leaders’ assessment of its pace, so an enterprise can have started work and still consider itself behind.

But progress and workload readiness measure different things, so investment planning needs to address both. A progress assessment describes how far technology changes have advanced; a readiness assessment asks whether the resulting systems can support a particular deployment. An established modernization program can therefore coexist with concern about mission-critical agents. CIOs need to connect work already underway to the requirements of their intended AI uses.

Set the scope around the business goal

Connecting those requirements to current work helps CIOs decide the order of a broader modernization program. Most enterprises are pursuing end-to-end modernization, meaning changes across their technology environment, but they can choose a narrower set of systems for the next investment. The eventual scope describes how much will change over time. The next project needs a business reason for each system it includes.

Chohan made this distinction in advice to CIO Dive, recommending that CIOs modernize legacy technology in stages. The starting point is the business’s intended use of AI, followed by identifying the systems that prevent that use. An older system enters the next project because it blocks the required capability. Other systems can stay in the wider modernization program until their place in the sequence becomes clear.

Chohan connects this staged approach to the organization’s ability to expand AI use: “By doing so methodically, they can build the foundation their organization needs to scale AI,” Chohan said. The proposed scope may still be substantial, but each change has a stated connection to the capability the business intends to deploy.

Use organizational knowledge to guide tool decisions

The connection between purpose and infrastructure must also guide CIOs as they consider additional tools. Chohan says continued AI investment will require leaders to sort through excess AI information and clarify the technology’s role in their organizations. A clear role helps them select a tool, and that choice creates a concrete planning task: assess the technical barriers between deployment and the business goal. Tool selection and modernization planning therefore inform each other.

Because those barriers depend on the tool and its intended use, each proposed deployment needs its own assessment. The selected tool may require changes to the systems identified earlier, so leaders need to assess those technical barriers before deciding which work to prioritize. The deployment assessment must address cost, speed, safety and whether the proposed changes are enough.

Making those assessments depends on understanding how the organization operates. “Few people know more about how an organization runs than its tech leaders – how it’s built, how its employees work, and how its customers experience its services,” Chohan said. That knowledge helps CIOs judge how a proposed tool would affect employees’ work and customers’ experience. It also helps them identify which technical barriers matter to the business result.

Chohan connects this understanding directly to solution design: “As a result, leaders are better equipped to build solutions that are shaped by their business needs rather than by market noise.” For CIOs, applying that knowledge means testing competing claims against the organization’s operations and translating relevant requirements into tool choices and system changes. Market attention can prompt a review, but the operational need determines which capability deserves investment.

Key executive takeaways

Assess AI infrastructure demands: CIOs need to account for computing capacity, data access, ongoing spending and security governance when planning AI investments. Identify which underlying systems each deployment requires changing.
Measure readiness by workload: Only 17% of IT leaders in Google’s report expressed confidence that their technology stack could support mission-critical AI agents. Assess modernization progress and readiness for each intended deployment separately.
Sequence upgrades around business goals: CIOs can stage modernization by identifying the legacy systems that block a specific AI capability. Give each system in the next project a clear connection to the intended business outcome.
Connect tool choices to operational needs: Evaluate proposed AI tools against employee workflows, customer experience and technical barriers. Use cost, speed, safety and the adequacy of planned system changes to set investment priorities.