Why AI search may see less of your website

AI search changes the objective

Website traffic is becoming a less complete measure of commercial influence as AI systems mediate research. Contentsquare, an experience analytics platform with a commercial interest in how companies analyze digital journeys, reported that AI-referred traffic to brand websites increased 632% in roughly 10 months. Jane Austin, SVP of design at Contentsquare, cited Forrester as finding that 51% of software buyers now start their research in an AI chatbot rather than a search engine, up from 29% the year before.

The traditional search model links several events: make pages understandable to a search engine, rank for useful queries, bring people to the website, and measure their activity. AI-mediated research can separate those events. An agent can collect product information, compare alternatives, and summarize a brand before a prospective customer visits its website. A company can therefore influence research without receiving a referral session from every interaction.

That changes the objective of optimization. Brands need accurate product and brand information wherever AI systems perform discovery, along with ways to determine whether that exposure contributes to demand and sales. The task is to preserve brand meaning and commercial measurement as more research occurs outside the website.

A site Google can understand may still be opaque to an AI agent

The first problem lies in the architecture of many modern websites. With client-side rendering, the server can initially return minimal HTML while JavaScript inserts useful text and interface elements in the visitor’s browser. A person sees the finished page because the browser executes that code. A system retrieving only the initial HTML may receive much less product information.

“The browser gets an empty shell, and then you render,” Austin said. AI bots “do a plain fetch of the raw HTML. They don’t wait for the page to build. There is nothing for them to read.” The distinction matters because a marketing team can see a complete product page while an automated system retrieves a materially different representation.

Google has historically been able to process JavaScript through rendering infrastructure based on headless Chrome. The durable engineering question is what each important machine consumer can retrieve from the site today. If one automated system renders JavaScript and another does not, they can receive different information from the same URL. Existing search rankings then provide limited evidence about what another system can retrieve.

This can create a gap between conventional search visibility and AI accessibility. For executives, the issue is how information reaches different machine interfaces. Austin recommends testing important URLs with curl or an appropriate bot simulator, then inspecting the returned HTML. Because Contentsquare sells experience analytics, its recommendations about auditing and measuring digital traffic come from a company with a commercial stake in this area.

Teams can apply the test directly: check whether product names, differentiators, specifications, pricing and supporting copy are present before browser-side JavaScript runs. The result shows what a simple retrieval request can see. Server-side rendering can address this retrieval problem by assembling page content on the server and returning useful HTML in the initial response. It makes important information available without requiring the requesting system to execute application code.

The executive question for engineering is concrete: when an automated system requests an important commercial page, which decision-relevant facts are present in the response it receives? That question turns AI accessibility into something teams can inspect rather than infer from conventional search performance.

Machine-readable HTML is only the beginning

A complete HTML response solves access to information encoded as useful text. It may still fail to communicate everything the human experience conveys. A product demonstration can show how an item looks in use, while an interactive module can reveal choices through user actions. Images can also carry material distinctions that sparse surrounding copy never states.

Andrew Frank, distinguished VP analyst at Gartner, calls this the “dual-mode media” challenge. His approach gives an asset a human-facing presentation and a machine-oriented layer containing material such as transcripts, chapter headings, alt text and structured metadata. Structured metadata is labeled information that describes content in a predictable form. These elements can preserve commercially important meaning when software encounters a different representation from the one a person experiences.

A product video shows the problem. A human can infer details from demonstrations, visual sequences and objects that appear without being discussed aloud, while an agent working from a transcript receives only the available words. “AI is likely to pick up on nuances of the semantic presentation that are perhaps invisible to people,” Frank said. “People don’t usually read the transcripts of a video. If the transcript has descriptions that are not in the video, there is an opportunity to replace some of that lost information.”

Transcript quality therefore affects brand representation. The same principle applies to images: useful alt text states relevant information about the image. Headings expose how information is organized, while structured metadata can make important attributes explicit. A practical test is whether the machine-oriented material preserves facts and distinctions that a buyer could otherwise learn from the visual or interactive experience.

Information-rich publishing practices can help. Comparison tables, specification sheets, FAQ sections and pricing pages state facts explicitly in forms automated systems can retrieve and interpret. These formats make complex evaluations easier to represent in text and structured information. Visually led experiences can leave more meaning inside photography, video or interactions.

Accuracy also affects the commercial value of AI-mediated discovery. “What we used to think of as SEO success is changing. It’s not just about visibility anymore. It’s about the accuracy of how your brand is being portrayed in the AI summary,” Frank said. When an AI system mediates research, companies have a reason to make authoritative product facts and brand distinctions explicit enough for those systems to retrieve.

Measurement becomes the harder problem

Contentsquare reported that one unnamed financial-services client lost 40% of its organic search traffic to a single page while conversion on that page increased. The example shows only that traffic and conversion can move in different directions; it does not establish AI as the cause. AI-mediated journeys can also produce different observable paths. A person might ask a chatbot to research options and later arrive at a company site through another channel.

An agent might instead retrieve information without producing a human visit. In either case, web traffic captures only the events that reach the company’s observable systems. Identity makes attribution harder. Austin said Cloudflare and other security tools can categorize traffic as bot or human and distinguish crawlers, live agents and scraping bots.

“That human binding agent is ultimately invisible,” Austin said. “The website treats that agent like a person, but there isn’t a fix for that identity gap.” That distinction matters because overall automated traffic is a different category from AI-mediated shopping or research. According to the Imperva Bad Bot Report, automated traffic accounts for 53% of all web traffic, and 40% of that automated traffic is malicious. A crawler indexing public information, an agent researching for a buyer and a malicious scraper can all produce automated requests with different commercial meanings.

That identity gap weakens a familiar attribution chain. An agent could collect information for a prospect who later arrives directly or through another channel, leaving analytics unable to establish that both activities belong to the same purchase journey. The prospect might also complete the journey without visiting the site. In these cases, event-level analytics can miss or misallocate the influence of information consumed through an intermediary.

Frank describes the more extensive scenario: “If you can do the whole journey with a chatbot that ends with a transaction, the role of the website is highly diminished.” Gartner projects that $15 trillion of B2B spend will flow through AI agent exchanges. If transactions increasingly occur through agents, website referral measurement would capture a smaller share of the purchase process.

Marketing measurement therefore has separate jobs: test whether AI systems represent products accurately, look for evidence that AI-mediated discovery contributes to demand, and connect that influence to commercial outcomes where possible. AI-referred conversion rates offer one signal when referrals remain observable. They cannot capture an interaction that affects a buyer without creating a traceable referral.

This changes budget decisions. Declining organic traffic does not by itself establish declining commercial influence, just as growing AI referrals do not establish accurate product representation. Leaders need evidence linking observable machine activity with demand and economic results before changing spending. That shifts management attention from session counts toward the commercial outcomes those sessions were intended to represent.

Marketing and technical teams need a two-part AI-discovery agenda

The first agenda is information availability. Brand and marketing teams can identify the product details, pricing, comparisons, support information and brand claims that matter to customer decisions. Web, SEO and engineering teams can then test whether automated systems retrieve that material. Server-rendered HTML, transcripts, useful alt text, clear headings, comparison tables, FAQs, specification sheets and structured metadata can expose the relevant information.

Teams can begin with focused experiments. Austin recommends auditing rendering with curl or a bot simulator; because this advice comes from Contentsquare, leaders should evaluate it alongside the company’s commercial interest in digital experience analysis. Austin also places API exposure and MCP server architecture within the engineering conversation. These tests give technical teams a direct way to find gaps between the information intended for buyers and the information machines can retrieve.

The second agenda is measurement. Analytics teams can establish a baseline that separates observable human and automated activity where their tools permit it, then examine conversion behavior by identifiable source and traffic type. Leaders can also test whether material brand facts appear accurately during AI-mediated discovery and compare those observations with downstream demand and sales signals. The aim is to distinguish changes in measurable referrals from changes in commercial influence.

Austin’s shopping example shows why representation deserves its own test. She described using an AI interface to search for a specific pair of gold hoop earrings with sapphires and receiving results without being able to judge which brands were high quality. “The brand signals and the trust signals are still needed,” she said. “You have to ensure that your brand appears, that it feels on brand, and that the experience of shopping still feels good.”

Ownership spans brand and marketing, web and SEO, analytics and engineering because each function controls a different part of the system. Brand teams define the distinctions that need to survive machine mediation. Technical teams control how much information machines can retrieve, while analytics teams determine what the company can observe. Marketing leadership then decides which measured outcomes justify investment.

Key executive takeaways

• Optimize for AI-mediated discovery: AI systems increasingly research and summarize products before buyers reach a website, weakening traffic as a complete measure of influence. Marketing leaders need to track accurate AI representation alongside demand, conversion and sales outcomes.
• Test what AI agents can retrieve: JavaScript-heavy pages may expose far less information to AI agents than they show to people or Google. Engineering and web teams should inspect important URLs and ensure product facts, pricing and differentiators appear in retrievable HTML.
• Make product meaning machine-readable: Videos, images and interactive experiences can contain information that automated systems miss. Brand and content teams should use transcripts, descriptive alt text, clear headings, specifications and structured metadata to preserve commercially important details.
• Expand measurement beyond referrals: AI-assisted research may influence purchases without creating a traceable website visit, making session-level attribution less complete. Analytics teams should separate identifiable traffic types and connect AI discovery signals with downstream demand and sales where possible.
• Coordinate AI discovery across functions: Brand, engineering, SEO and analytics each control part of how AI systems retrieve, represent and measure company information. Marketing leadership should establish shared tests for information availability, brand accuracy and commercial impact.