Customer-facing AI needs review when its role changes

Customer-facing AI approval needs renewal

A customer-facing AI system can keep its original approval even after its work has changed enough to require a new decision. An assistant approved to answer questions may begin recommending outcomes, gain access to customer accounts or leave an employee little practical chance to disagree. Approval remains useful while its assumptions about authority, customer outcomes and human control still hold.

Those assumptions matter across familiar uses: generating marketing content, personalizing offers, answering customer questions and recommending what people should consider next. Each can affect a customer through the information presented, the data collected or the action taken. So governance has to follow the system after deployment, alongside the testing and legal review required before launch.

Customer trust makes that continuing responsibility more urgent. Security vendor Thales, which benefits commercially from demand for data protection and security, reports in its 2026 Digital Trust Index that only 23% trust companies to use AI responsibly with their data. For teams preparing or reviewing a system, that concern raises a practical question: what authority has the organization granted, and how does it control its use?

Define the system’s authorized actions before launch

Authority requires a separate decision from technical capability. An AI agent, meaning a system connected to tools that can perform tasks, may produce a factually correct answer and then trigger an action outside its intended authority. Evaluating answer quality therefore needs to go alongside checking the business systems it can reach and the restrictions applied while it operates.

These surrounding controls are central to the warning Allan Dabre, senior manager of technology compliance and AI risk at PwC, gave CMSWire. PwC sells advisory services in these areas and benefits from demand for AI risk management. Dabre said: “The risk teams miss most often sits in the harness, not the model. Companies spend months evaluating a vendor’s training data and bias testing, then hand the resulting agent broad system access with almost no runtime controls on what it can do once deployed.” Here, harness means the systems, permissions and controls governing a deployed agent; runtime controls enforce restrictions while it operates.

Broad access has consequences when an agent can issue a refund, change an account setting or escalate a complaint without anyone having approved that permission. Each action may be legitimate within customer service, but each requires an explicit decision about independent authority. A refund policy, for example, needs to specify when an agent may act and when a person must approve the outcome. The technical connection should enforce that decision.

Writing down those permissions is the starting point proposed by Emily Hartstone, founder and CEO of From The Hart Management, whose business advisory work gives her a commercial interest in demand for governance advice. She advises businesses: “Start with the question most governance skips: what is this system allowed to do without a human. Not what can it do, what is it authorized to do. Write that list before launch, because if you can’t write it, you don’t yet have a system you can govern. You have one you’re hoping behaves.” The list gives product, engineering and risk teams a shared basis for approving connections and actions.

Enforcement turns that agreement into an operating restriction. Teams need to identify connected business systems, specify access permissions, define permitted actions and establish escalation rules. Training-data examination and bias testing remain necessary, while runtime controls prevent an accurate response from triggering an unauthorized consequential action. Records should identify who approved those limits, and customers need a route to challenge consequential decisions.

Reassess approval when use or human control changes

Maintaining those limits starts with finding what is already operating. In AI governance software vendor OneTrust’s 2026 AI-Ready Governance Survey, 1,200 senior decision-makers across eight markets reported a gap between encouraging AI agents and governing them. OneTrust has a direct commercial interest in that gap because it sells software for managing governance.

The survey found that 87% of organizations encouraged AI-agent use, while 47% had clear governance controls and oversight for those agents. Thirty-one percent identified AI use cases only after they were already operating. When discovery follows deployment, the first governance decision may arrive after employees and customers have begun depending on a system. An inventory therefore needs active discovery alongside registration before launch.

Once a deployment is identified, reviewers need to check whether human oversight still changes decisions. Dean Cooper, group IT director and creator of the Human-Accountable AI Responsibility Model, described the progression to CMSWire: “A customer-service AI might begin by helping an employee draft a response. Then it starts analyzing the customer’s request. Then recommending an outcome,” he said. Each step gives the system more influence over the judgment that follows.

An employee can stay involved throughout that progression while losing practical control. Cooper continued: “Eventually the employee may effectively be approving a decision the AI has already made. On paper, the organization can still say there is a human in the loop. Operationally, that human may have become little more than the final click.” So an approval record needs to be considered alongside evidence that the employee examined the recommendation and could challenge it.

Meaningful review depends on the employee’s working conditions. The reviewer needs enough information to assess the recommendation, enough time to investigate concerns and authority to reject or change the proposed outcome. As the AI’s role expands from drafting to analysis and recommendation, teams should check whether employees actually evaluate decisions. An approval screen supports accountability only when the reviewer can exercise judgment through it.

Changes in the system’s purpose require a similar reassessment. Usama Ahmad, founder of ReceptFlow, has a commercial interest through his company in how businesses adopt and manage AI. He describes the risk: “An AI system approved for ‘answer FAQs and book appointments’ quietly starts handling things like symptom triage or legal-intake screening because nobody re-reviewed it as its usage drifted.” The original approval then covers work with different consequences from those initially considered.

As responsibilities expand, teams should update the inventory and reassess changes in data, capabilities, users and business purpose. Moving from appointment booking into symptom triage, for example, changes the expertise and safeguards needed to handle an incorrect response. Reviewers should check whether the restrictions, human oversight and customer support routes still fit. Renewal should follow material changes in authority, observed outcomes or practical human control, rather than a universal calendar interval.

Test answers and outcomes in the conditions customers face

Reassessing authority is one part of review; checking what customers receive is another. In February 2024, British Columbia’s Civil Resolution Tribunal held Air Canada liable in Moffatt v. Air Canada after its website chatbot supplied incorrect information about bereavement fares. The tribunal rejected the argument that the chatbot bore independent responsibility for its statements. The company’s liability arose from information supplied to a customer, which means account-access restrictions and continuing accuracy checks address different kinds of failure.

Outcome testing must also examine how automated decisions affect different groups. In May 2025, a federal judge in California allowed claims in Mobley v. Workday concerning alleged age discrimination by Workday’s AI applicant-screening tool to proceed as a nationwide collective action. The claims concerned older job seekers; the procedural decision allowed the case to advance without deciding whether discrimination occurred. Although applicant screening serves a different audience from customer service, it illustrates why reviews of consequential decisions need to examine treatment across affected groups.

Testing a particular deployment begins with the questions its users actually ask. JP Garbaccio, head of AEO, or answer engine optimization, at Searchable, works in a commercial field concerned with how organizations appear in AI-generated answers and has an interest in demand for evaluating those outputs. He told CMSWire: “Training data tells you what a model learned from, and a vendor’s assurance tells you how it did on their questions. Neither tells you what it will say to your customers.” Acceptance tests therefore need to reflect the company’s products, policies and customer records.

Garbaccio recommends a specific sequence: “So test it with real questions from your support logs, run each one many times, and mark the answers against facts you have checked yourself. Break the error rate down by relevant segments, such as product and region.” Repeated runs expose variation in responses to the same question. Scoring against checked facts establishes whether those responses are acceptable for the business, while segmenting results shows where errors cluster.

The same examination should cover fairness in customer treatment. Teams should compare error rates, recommendations, prices, service access and decision outcomes across relevant groups, including realistic edge cases. Different speech patterns, incomplete records and unusual circumstances can expose weaknesses in standard workflows. Testing these conditions helps identify who receives weaker answers or less useful service, while documented investigation and correction procedures establish what happens when a disparity appears.

Live operation adds conditions that controlled testing can miss. Complaint patterns, escalation rates and differences in customer outcomes can reveal recurring barriers, including situations a human employee could recognize and resolve. When those signals appear, teams should review model behavior, prompts, the instructions supplied to the model, and connected information sources, then correct the problem or pause operation. Vendor-model updates and changes to the information supplied to the system should also trigger retesting.

Judge customer benefit by relevance, fairness and usable recourse

Testing can establish accuracy, but customer benefit also depends on whether people can resolve their problems. In a Gartner survey of 3,566 business-to-business (B2B) and business-to-consumer (B2C) customers conducted in February and March 2026, 50% said interactions were easier when companies used generative AI, or GenAI, which produces content such as text. In the same survey, 87% said companies using GenAI for customer service must offer a human-agent option. Gartner sells research and advisory services, so it benefits commercially from demand for guidance on these choices.

The combination of convenience and human access points to a practical design requirement. When an issue is sensitive, complex or unresolved, or an action has meaningful consequences, customers need a route to someone who can exercise judgment, explain a decision and take responsibility for the next step. Complaints, escalation, human support and appeals provide different forms of recourse. Their value depends on whether they help the person resolve the issue, including when their circumstances fall outside the standard process.

Expectations also vary with the use of AI. A Gartner survey released in March 2026 found that half of U.S. consumers preferred to give their business to brands that did not use GenAI in consumer-facing messages, advertisements and content. Separately, a Gartner survey of 1,539 U.S. consumers, conducted in October 2025, found that 68% frequently wondered whether the content and information they encountered was real. These findings concern brand preference and uncertainty about authenticity, distinct questions from convenience and human access in customer service.

For marketing teams, those concerns make relevance a necessary test of personalization. Stefanie Boyer, professor of marketing at Bryant University, told CMSWire: “We see this in sales outreach. Companies can now generate highly personalized emails at enormous scale, but ‘personalized’ does not mean relevant. An AI might reference a conference someone spoke at six years ago and then jump immediately into a generic sales pitch. Technically, it is personalized. From the buyer’s perspective, it can actually feel less authentic and damage trust.” Her example shows how using a personal detail can fail to address a current need.

A customer’s current need gives teams a practical basis for judging the interaction. An airline cancellation alert with a rebooking option addresses a known problem; a retailer using inferred vulnerability to pressure a purchase exploits sensitive knowledge. Engagement and conversion improvements should remain secondary to whether the interaction helps the customer. Review should therefore cover relevance, unsupported claims, sensitive inferences, misleading synthetic content and provenance, meaning the content’s origin.

Disclosure helps customers understand those interactions, while the underlying practices determine whether they deserve trust. Companies should identify AI involvement when relevant to answers, recommendations and content that influences decisions, and plainly explain the system’s source, limitations and important constraints. They also need meaningful consent for unexpected data uses and should collect only the information required for the task. Collection, retention or exposure beyond reasonable expectations can undermine an otherwise useful service.

Privacy, choice and recourse therefore belong in the assessment of customer value. A relevant recommendation can still depend on inappropriate data use, while an accurate answer can leave a sensitive issue unresolved if no qualified person is reachable. Legal review adds requirements to these product decisions, with duties that depend on jurisdiction, data use and the consequences for individuals.

Map legal duties to the use case and its changing consequences

Legal review needs to connect each deployment to where its customers are, the data it uses, the claims it makes and its potential consequences. As of October 2026, the EU framework includes obligations already in force alongside later deadlines for other requirements. Teams therefore need to assess current duties separately from preparations for later compliance dates.

Current duties include the EU AI Act’s prohibited-practice rules and a revised staff AI-literacy obligation, concerning the knowledge staff need to understand and use AI. The Digital Omnibus on AI, Regulation (EU) 2026/1744, requires companies to take measures supporting staff AI literacy rather than guarantee a set level. Its timetable also separates transparency requirements from later obligations for high-risk systems, so the deployment’s use and relevant deadline are important parts of review.

Against that timetable, EU-facing teams should assess interaction disclosures, relevant content labels and machine-readable marking, which allows software to identify generated material. They also need to consider the General Data Protection Regulation, or GDPR, and its restrictions and safeguards for solely automated decisions producing legal or similarly significant effects. Review should examine personal-data processing, consequential decisions, content requiring identification and risks to fundamental rights. Later AI Act deadlines leave existing obligations in force.

The United States requires a different review because it has no comprehensive federal AI statute. Federal consumer-protection law, including law enforced by the Federal Trade Commission, or FTC, still applies to deceptive claims, unfair practices and impersonation. Companies hosting or distributing user content should also consider the Take It Down Act and rules addressing synthetic intimate imagery. The Act covers certain AI-created or altered intimate imagery.

State requirements add obligations concerning notice, profiling, sensitive information and automated decisions. California Privacy Protection Agency, or CPPA, regulations on automated decision-making technology, also called ADMT, require compliance by Jan. 1, 2027. Colorado replaced its 2024 AI Act with the narrower automated-decision transparency law SB 26-189, which takes effect Jan. 1, 2027, subject to a pending legal challenge that could delay it. These differences require teams to map duties to each use.

Legal challenges also need careful handling when teams record current obligations. Executive Order 14365, signed Dec. 11, 2025, established a Justice Department task force to challenge state AI laws, while leaving those laws in place unless another action changes their effect. Teams should distinguish an obligation’s effective date from a challenge that may affect it. That record gives deployment owners a basis for responding when legal conditions change.

Assign an owner and a review process for the full lifecycle

Responding to legal and operational changes requires one named owner for each deployment. Product teams define its purpose, while data, security, legal, compliance and customer-facing teams oversee access, obligations and communications. Leadership must require detection of errors, bias and customer harm after launch. The deployment owner coordinates review and escalation so responsibility stays clear when several teams need to act.

The owner needs a deployment record that supports decisions throughout the system’s lifecycle. It should identify the purpose, affected customers, data access and whether the system influences actions or performs them. It should also document risk classification, approved and prohibited applications, important data and model choices, and the reasoning behind significant design decisions. Reviewers can then compare current operation with the decisions that justified approval.

Creating that record begins with a narrow use case and an account of unacceptable harms. Before deployment, teams should assign the owner, identify affected customers and establish review and escalation routes. Data preparation and design should then follow those limits: record the necessary information, restrict access, document model and design decisions, and specify allowed and forbidden actions. This connects the stated purpose to limits on independent authority.

Before launch, teams should test whether the design meets those requirements under realistic conditions. Accuracy, security and fairness testing should include unusual cases and differences between groups, alongside checks that human escalation works. Teams should also define conditions for rollback, meaning a return to an earlier, safer system version or configuration. After launch, complaints, errors, disparities, security weaknesses and model changes should feed into monitoring, inventory updates, correction and reassessment.

The intensity of that work should increase with potential harm. Credit, employment, insurance, housing and access to essential services can affect opportunities or basic needs, so scrutiny must reflect those consequences. Customer-service and marketing deployments also need specific controls because wrong information, inappropriate data use and manipulative targeting can harm customers. The use determines which safeguards need particular attention.

These safeguards become usable through intervention procedures agreed before problems arise. Teams need defined conditions for pausing an AI-enabled process, procedures for correcting faulty outputs and criteria for rolling back risky changes. Continuing security review must follow pre-launch testing, and someone must retain the ability to explain, change or stop each deployment. For customers outside standard workflows, those procedures must preserve a genuine opportunity to contest consequential decisions and reach qualified help for sensitive or unresolved matters.

The bottom line

Approval for customer-facing AI rests on assumptions about what the system may do, who it affects and how people retain control. Those assumptions can stop holding as teams add capabilities, connect more data or rely more heavily on recommendations. Leaders should treat material changes as decisions requiring review, rather than letting the original approval cover an expanding role.

That requires a named owner with authority to intervene, evidence of customer outcomes and clear conditions for reassessment. Review should establish whether permissions still fit the task, employees can genuinely challenge recommendations and customers can reach someone able to resolve consequential problems. An approval record alone cannot answer those questions.

The executive responsibility is to fund and enforce those controls throughout operation. Teams need time to investigate disparities, correct errors and pause systems when safeguards no longer fit. Customer-facing AI remains accountable only when the business can explain its actions, limit its authority and take responsibility when it causes harm.