Why customer reliance determines the value of agentic AI

The most capable AI still fails if customers will not rely on it

A customer-facing AI can make a fast, sophisticated decision and still produce little business value if customers refuse to rely on it. Accuracy, speed and efficiency matter. Their commercial value depends on whether people use the system for the work it is meant to perform.

This matters more with agentic AI: systems that can pursue goals, make decisions and take actions with a degree of autonomy. Executives face a direct management problem: if customers decline to rely on an AI system, technical capability alone cannot deliver the intended customer outcome.

Trust is an operating requirement

A system that produces inaccurate answers, fails unpredictably or takes inappropriate actions has a performance problem. Reliability and accuracy are basic requirements for consequential customer use.

Technical performance is only part of a customer’s decision to rely on a system. Customers also encounter its explanations, decisions and changes in behavior. Each consequential interaction gives them new information about how the system operates.

This makes customer reliance an operating concern. Teams need measures of quality and performance, along with processes for explaining consequential outcomes, recording what happened and communicating meaningful changes.

For executives, the management question extends beyond whether a system passed its checks before deployment. The organization also needs to know whether it can explain and reconstruct important decisions after deployment.

Complex customer interactions expose the acceptance gap

EY Studios has stated: “Customers will have a bias for human-to-human interaction when dealing with a complex interaction. As we design these services, we’ll need to think about how we overcome this bias and train customers to use it effectively.”

This is EY Studios’ characterization of customer behavior. It identifies a practical design question: an organization may be able to automate an interaction while customers still prefer human handling.

Consequential decisions raise the stakes. A customer may need to know what information influenced an outcome and how the system used it. The organization may also need a process for reviewing disputed or unexpected outcomes. Clear explanations and access to appropriate review give customers a way to assess what happened.

Explainability and auditability serve different needs

Explainability starts with the customer. For a consequential decision, an organization can provide a plain-language account of the relevant information, how it affected the process and which rules led to the outcome.

Quant, which specializes in deploying agentic AI, describes this as a foundation of its own approach: its systems are designed to show clients the data used by an agentic AI, how the agent used that data and the rules behind its decision.

Quant has a commercial stake in this framing because it specializes in deploying agentic AI and benefits when organizations invest in such systems. Its description is a vendor account of its own practice.

The internal requirement is different. Organizations can preserve records that allow teams to reconstruct important AI-driven decisions. An audit trail is the record of the data, applicable rules, actions and decision path needed to trace what happened.

The distinction matters. Customers need explanations they can understand. Internal risk, compliance and operations teams may need a more detailed record to investigate an outcome. Raw technical logs can make a customer explanation harder to use. A simplified customer explanation may lack the detail needed for an internal investigation.

Communication adds another layer. A deployment or system change can alter how customers encounter a service. When that change materially affects the customer experience, teams should explain what is changing and how it affects users.

The same principle applies to reporting. Teams should understand the end user’s goals and describe results in terms the intended audience can interpret. This keeps communication tied to the decision or service the customer actually experiences.

Customer reliance belongs to the operating model

Responsibility crosses several functions. Leadership sets risk limits and determines which outcomes require explanation. System designers determine whether decisions can be reconstructed. Customer-experience teams design explanations and review paths. Operations teams preserve records and communicate service changes. Compliance teams use traceability when their work requires it.

Executives can apply a practical test to consequential AI actions: can the organization explain the outcome to the affected customer, reconstruct what happened internally and support an appropriate review?

The test also applies when the deployed system changes. Change-management processes can identify upgrades that materially affect customers. Documentation should describe the deployed system, while customer communication should explain the effects users will experience.

This puts governance around the full service surrounding an AI decision: the action, its explanation, the underlying record, the review process and communication about material changes.

Evidence and the business case

EY Studios’ statement identifies a claimed preference for human interaction in complex cases. Quant describes an approach based on showing clients the data and rules behind agentic AI decisions. These claims address different parts of the problem.

Executives should separate two questions in measurement. First, do consequential AI decisions remain understandable and traceable? Second, do specific interventions change adoption, retention or other commercial outcomes? Testing these outcomes separately gives executives evidence for deciding which operating practices improve customer reliance.

Main highlights

• Trust is an operating requirement: Leaders should treat customer reliance as a condition for realizing value from agentic AI. Measure technical performance while ensuring consequential decisions can be explained and reconstructed after deployment.
• Complex interactions test customer acceptance: Customers may still prefer human support for complex interactions even when AI can automate them. Provide clear explanations and appropriate review paths for consequential or disputed outcomes.
• Explainability and auditability serve different needs: Give customers plain-language explanations while maintaining detailed internal records of relevant data, rules, actions and decision paths. Do not treat customer-facing explanations and technical audit trails as interchangeable.
• Customer reliance belongs to the operating model: Assign responsibility across leadership, design, customer experience, operations and compliance. Governance should cover AI actions, explanations, records, review processes and communication about material changes.
• Measure the business case separately: Test whether AI decisions remain understandable and traceable, then separately measure whether interventions affect adoption, retention or other commercial outcomes. This helps leaders identify which practices actually improve customer reliance.