What CX leaders must decide before customer service becomes autonomous

A successful AI interaction can still fail the customer

An airline AI agent can follow its rules and still make a decision the airline should have reserved for a person. Consider an illustrative case: a cancelled flight threatens a passenger’s attendance at a parent’s funeral. The agent checks the fare conditions, finds an alternative, calculates compensation and completes the interaction within minutes. Its speed and procedural accuracy can count as strong performance even though the passenger’s circumstances require discretion.

The failure comes when a permitted resolution becomes the final decision about how the customer will be treated. The rules may establish what the agent can offer while leaving open what the airline owes someone in an emotionally consequential situation. For leaders responsible for customer experience, or CX, this raises a question of authority: how much control should automation have over the customer promise, the commitments and principles that define the organization’s intended relationship with its customers?

AI adoption makes decision rights more urgent

The authority question grows more urgent as AI takes on decisions previously made by employees. EY India’s Responsible AI Pulse Survey found that 37% of CXOs, or senior executives, struggle to develop AI governance frameworks that keep pace with technology, while 73% of surveyed organizations report AI embedded in key initiatives. Adoption is substantial even as organizations report difficulty governing it. EY provides advisory services and has a commercial interest in organizations investing in AI governance, which readers should keep in view when considering its findings and recommendations.

For organizations facing that difficulty, governance must cover both technical risks and authority over customer treatment. Controls for hallucinations, plausible but incorrect AI-generated statements, help assess whether answers are dependable, while customer-data protection, bias controls and explainability address how systems handle information and reach results. These controls are essential. Alongside them, leaders need to specify which aspects of the customer relationship a system may interpret, change or negotiate.

Those permissions matter more as customer-facing AI moves from answering questions to recommending, deciding, negotiating, resolving and acting for an organization. An incorrect answer can mislead a customer, while an authorized system can carry out an incorrect decision and commit the business to an action. Execution therefore raises the stakes of an error. Accuracy and authority require separate judgments because producing a sound answer does not establish permission to make the decision that follows.

Performance measures shape customer treatment

Authority becomes harder to govern when an organization treats performance against its chosen measures as evidence of appropriate customer treatment. A key performance indicator, or KPI, expresses a result the organization values; making that result an optimization objective encourages behavior that advances it. Applying the same objective across many interactions turns an internal management preference into the service customers receive. Leaders therefore need to examine what each measure rewards when a difficult case arrives.

Containment shows how this happens in customer service: it measures whether an interaction stays within automated service rather than moving to a human. Giving containment heavy weight rewards the system for avoiding escalation, including when a person should intervene. A bereaved customer may need an exception that the ordinary process cannot grant, yet the system’s objective may count access to a person as an unsuccessful outcome. The interaction can become inappropriate while the system behaves exactly as its incentives encourage.

Conversion creates a different pressure in recommendation systems because it measures whether recommendations lead to purchases. When conversion becomes the main objective, advice may increasingly favor what produces a sale, weakening recommendation neutrality. A customer relying on the organization’s judgment then receives advice shaped more strongly by its commercial goal. A strong conversion result can conceal that shift.

Resolution time creates a related pressure by rewarding quick case closure. A system pursuing that objective has a reason to favor cases that fit established procedures, while customers whose circumstances require more consideration become harder to serve within the target. The measure can improve as service becomes less accommodating to people who need time and judgment. These examples show why leaders need to assess an objective through the customer treatment it encourages.

The customers most exposed are those whose circumstances make standard treatment inadequate. Bereavement, financial distress and medical vulnerability can change what an appropriate response requires, while exceptional loyalty may justify consideration beyond the ordinary rule. An emotionally consequential complaint may also require someone to understand why an available resolution remains insufficient. Because discretion takes attention, governance centered on efficiency can overlook the customers who most need it.

For these customers, human intervention belongs in the service design. Access to a person authorized to exercise judgment may be part of what the organization promises, so escalation delivers that commitment. Treating every handoff as a cost or failure can remove a form of value customers reasonably depend on. The measure should therefore allow for situations in which reaching a person is the appropriate outcome.

Allowing for those situations still leaves containment, conversion and speed as useful goals. Faster service can be valuable, automated resolution can be appropriate, and a recommendation can serve both a customer and a commercial objective. Governance determines how much weight each goal receives and what happens when it conflicts with another obligation. Performance against a measure establishes that the selected result was achieved; judging its quality requires attention to the customer’s circumstances.

CX leaders already face this incentive question when managing employees. Informal peer discussions among CX leaders have raised whether employee measures encourage appropriate behavior, and machine objectives deserve the same scrutiny. An audit should examine how targets affect escalation, recommendation neutrality and complex cases, investigating these effects in each deployment. Examining the model alone can miss a problem created by the organization’s choice of objective.

Define AI’s permissions and human decision rights

An audit can improve objectives, but it also exposes a separate question about permission. An objective specifies the result a system should pursue, while decision rights specify which choices the organization authorizes it to make in pursuit of that result. Even a system with a well-designed objective can encounter a decision the organization wants a human to own. Better policies and KPIs therefore need explicit limits on delegated authority.

The proposed Customer Promise Boundary offers a way to record those limits. It is a governance layer that sets out what automation may improve, which principles must remain protected and which decisions require human authority. Its practical value would depend on how clearly an organization assigns permissions and responsibilities. Those assignments help teams decide what a system may do before a difficult customer case arrives.

Within that structure, an organization could permit AI to improve response time, route customers to appropriate help, personalize an interaction and make recommendations more relevant. Each permission would define where the system could adjust service delivery and within what scope. Customers might reach someone equipped to help more quickly or receive advice better matched to their circumstances. The permission would also state where the system must stop and seek another decision-maker.

Protected principles would set the conditions under which those improvements remain acceptable. Transparency, fairness and consistency are possible commitments to preserve even when another choice would improve an efficiency measure. Recording those commitments helps decision-makers assess whether pursuing an objective remains compatible with the intended customer relationship. It also makes the meaning of better service more precise.

Decisions reserved for people would then make clear where a human must take responsibility. Depending on its promise and service context, an organization might reserve situations involving empathy, material exceptions, sensitive judgment or consequential financial decisions for a human. This is a question of authority even where software could carry out the available action efficiently. In the airline case, this could mean authorizing routine alternatives and compensation while reserving a material bereavement exception for a human decision-maker.

Setting these permissions in advance makes access to discretion an established service commitment. A relatively stable customer promise can then provide continuity as models, platforms and capabilities change. One explanation for governance difficulty is that organizations try to govern changing technology before agreeing on the human principles they intend to preserve. Starting with those principles gives later technology decisions a common reference, while leaving room for interpretation as capabilities and customer situations develop.

These principles can fit within existing governance arrangements. The people responsible need authority to enforce optimization permissions, protected principles and limits on autonomous negotiation; a separate committee or department is unnecessary. Better KPIs remain essential within these limits. Decision rights establish who may act, and KPIs guide performance within the authority granted.

Give CX a formal role before expanding autonomous authority

Enforcing these limits requires shared ownership across the functions already responsible for AI. Technology, legal, security and compliance assess matters such as technical control, legality and protection. CX leadership adds responsibility for whether the system’s conduct stays faithful to the intended customer relationship. A formal CX role brings that judgment into governance while preserving the other functions’ responsibilities.

Without that role, technical and control teams can leave the customer perspective underrepresented. A decision may be secure, lawful and correctly executed while still producing treatment the brand would reject. CX participation creates a place to examine this gap before autonomous behavior becomes routine. It also gives protected customer principles an organizational owner who can challenge conflicting objectives.

The decisions reserved for people need equally clear ownership. A requirement for human judgment remains incomplete until the organization identifies who must exercise it. Escalation must connect the system’s limit to an authorized person, with accountability remaining clear after the handoff. Otherwise, the organization can formally reserve a decision while leaving the customer without anyone responsible for making it.

Clear ownership matters more as systems gain the ability to act. EY observes that agentic AI, systems able to take actions toward an objective, shifts governance from checking outputs toward ongoing supervision of behavior, with greater attention to intent, control, accountability and escalation. Intent concerns what the system is trying to achieve, while control concerns the limits placed on its actions. Those controls, together with responsibility and escalation arrangements, should be established before autonomy is granted.

Granting that autonomy requires a deliberate sequence. First, define the customer promise, then establish the parts AI may optimize. Next, identify principles that remain protected regardless of efficiency and determine where autonomous authority gives way to human judgment. Only then should leaders decide how aggressively to expand deployment, because scaling increases the consequences of the authority already granted.

Key takeaways for decision-makers

• Set limits on AI authority: A system can follow rules and still mishandle a consequential customer situation. Customer service owners need to identify where routine resolutions give way to human discretion.
• Govern decisions as AI gains autonomy: AI governance needs explicit permissions for systems that recommend, negotiate and act. Assign decision rights before granting autonomous systems authority to commit the business.
• Audit what service targets reward: CX leaders need to examine how containment, conversion and resolution-time targets affect customer treatment. Treat appropriate escalation as a successful outcome when customers need human judgment.
• Define a customer promise boundary: Record what AI may optimize, which customer principles remain protected and which decisions require human authority. Use these limits to guide deployment as technologies and capabilities change.
• Give CX formal governance ownership: Include CX alongside technology, legal, security and compliance in decisions about autonomous customer service. Connect every required escalation to a person authorized to decide and accountable for the outcome.