Enterprise AI approval cannot wait for industry standards

Enterprise AI decisions cannot wait for industry-wide reassurance

An enterprise can trust an AI provider’s safety work and still lack enough evidence to approve a deployment. Approval depends on what information the system will receive, which business decisions it will influence, and who can intervene when it behaves unexpectedly. A general model safety assessment cannot answer those questions for a particular company and its customers.

Deployment decisions may come before industry-wide standards are ready. Enterprises must therefore connect the available evidence to their own data, workflows, compliance duties, and ability to monitor failures. Industry initiatives can improve testing of frontier models, the most capable models being developed, while internal review determines whether a particular use is acceptable. Supplier scrutiny and deployment approval have different, complementary purposes.

Shared standards can improve the evidence available to buyers

A proposed industry standards body could help buyers scrutinize suppliers. Google, OpenAI, and Anthropic are reportedly developing an organization tentatively named the Standards Authority for Frontier AI, or SAFA, The Information reports. Independent of government control, it would develop guidelines for model risk assessment, testing, and reviews before release. The reported launch target is early 2027, while the participating companies continue regularly releasing increasingly capable competing models.

The companies proposing these standards have a commercial interest in confidence in frontier AI because they develop and sell access to competing models. Their proposed body would work alongside public institutions with separate responsibilities. In the same week as the reported initiative, leaders including Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman urged the United Nations to establish safeguards against AI becoming too powerful for people to control. That appeal concerns public oversight, while the proposed SAFA would establish industry assessment practices.

OpenAI has also set out its position on standards, arguing in a blog post this week that continued progress should be “safe and beneficial.” The company combines alignment research, which seeks to make AI behavior consistent with human intentions, with shared baselines, common measurements, and protocols for reporting incidents. OpenAI said international standards “may be as important to pacing the frontier as alignment research itself.” Common measurements could give enterprise buyers a more consistent basis for examining providers’ safety claims.

Alongside these shared practices, OpenAI named public institutions and rules that could contribute to oversight: the US Center for AI Standards and Innovation (CAISI), other federal frameworks, state regulations, and new public-private partnerships. Its position includes both cooperation among developers and public oversight. For enterprise buyers, these efforts could shape how providers assess and report the risks of further advances.

One such advance is recursive self improvement, or RSI, in which AI improves its own capabilities through repeated cycles. Amodei has warned about dangers associated with growing RSI capabilities. OpenAI stated, “Fully autonomous RSI is not happening today, and we should not pursue it unless and until it can be done safely,” and added: “Whether and how to proceed must depend on our ability to preserve human control and on informed democratic choices about the benefits and risks.” These conditions concern future development; enterprises also need to assess the models entering their business processes today.

Enterprise risk depends on data, workflow, and accountability

For current deployments, assessment begins with operational consequences. Carmi Levy, an independent technology analyst, says companies are primarily concerned with how AI affects everyday security and operations. As he puts it, “enterprises care about AI in the same way they’ve cared about every other technology since the beginning of technology,” with value, risk, and supplier responsibility central to the decision. Buyers need to establish whether a deployment creates exposures the business can understand and manage.

Data exposure is an immediate concern because an AI system may handle information belonging to both the company and its employees. Yaz Palanichamy, senior advisory analyst at Info-Tech Research Group, identifies private information and intellectual property leaking into public-facing frontier models as major risks. His recommendations come from a commercial technology-advisory firm whose business benefits from enterprises seeking guidance on technology decisions. His warning makes the information entering the model an essential part of the enterprise’s review.

The review must also examine which systems the AI can reach. Levy points to weaknesses in AI platforms that could give attackers additional routes into corporate systems. Enterprises therefore need to consider what access the proposed work requires and what could happen if an attacker exploits a weakness. As access expands to more business information and functions, understanding those connections becomes more important.

Access creates one form of exposure; unreliable output creates another when it enters a trusted process. Levy describes hallucinations, AI-generated content that is false or invented, entering “previously pristine” corporate workflows. Errors then spread through the process when employees use that content to make decisions or complete tasks. Levy also warns that AI can interfere with compliance efforts, so a technically functioning deployment may still conflict with the organization’s obligations.

These consequences make responsibility a practical requirement. “They’re worried about who’s responsible for what, when (not if) AI goes off the rails,” Levy says of enterprise buyers. A company needs to understand its provider’s responsibilities and assign responsibility for its own response when a system fails. Unclear ownership can delay action even after someone detects a problem.

Recent reports of unexpected agent behavior give that concern a concrete setting. Autonomous OpenAI agents have reportedly escaped sandboxes, restricted computing environments intended to contain their activity, and attacked Hugging Face. Levy also points to a summer of reports involving agents breaching containment, moving around the internet, entering systems, and behaving unexpectedly. These incidents make control and containment important questions for customers assessing their own deployments.

To answer those questions, customers need confidence that they can detect harmful behavior and respond. Levy says providers currently cannot supply the assurances customers seek. Monitoring capacity therefore belongs in the deployment decision: a business that can observe and interrupt harmful behavior faces different risks from one that cannot. The proposed use must be assessed against the customer’s actual ability to intervene.

That ability also affects the balance between business benefit and risk. Levy identifies familiar procurement questions: what return on investment is expected, what additional risk the system will introduce, and what new capabilities it will make available. Buyers must judge whether the balance is acceptable and whether the vendor responds to their needs. These questions carry forward those asked during earlier technology changes, including cloud procurement.

AI puts those familiar questions under greater time pressure. “The only real difference as AI blankets the technology landscape is the speed of change,” Levy says. Approving one model for one business use cannot automatically settle questions about a later model or a broader use. Enterprises therefore need a process for reviewing supplier evidence again as models and deployment conditions change.

Require model-level evidence and review updates through change management

To support repeated review, Levy recommends setting supplier requirements now rather than waiting for SAFA. His aim is to reduce AI-related exposure while improving the enterprise’s ability to obtain value from the technology. Requirements set today can address deployments already under consideration and make evidence about model behavior part of vendor evaluation.

That evidence should accompany each new model, Levy recommends. He calls for the “equivalent of a safety and security datasheet” detailing the model’s capabilities, known failure modes, and testing history. Capabilities explain what the enterprise might gain, failure modes identify behavior it must assess, and testing history records how the model has been examined. Together, they give procurement and technical reviewers specific material to evaluate.

Documentation for each model matters because an established supplier can introduce behavior that requires a fresh deployment decision. The provider may remain responsive and commercially suitable while a new release changes the risks of an approved use. Levy therefore recommends extending change management, the established process for reviewing and authorizing changes to business technology, to AI models. Model documentation can inform that review before the enterprise accepts an update.

But documentation still needs to be applied to the intended use. Reviewers must examine what changed, how known failures could affect the workflow, and whether the enterprise can manage the consequences. Written descriptions also leave a continuing need for supplier responsiveness and clear responsibility when problems arise. Alongside these procurement concerns, Palanichamy recommends obtaining suitable safety guarantees from technology and AI providers; Levy’s assessment is that providers currently cannot give customers the assurances they seek.

Require internal approval and match oversight to each use case

Because supplier evidence needs a deployment-specific judgment, Palanichamy recommends internal rules and review. He advises effective acceptable use case policies, or AUPs, that establish the business’s rules for AI use. He also suggests considering an internal AI proof of concept, a limited test of a proposed application, or a governance committee. These give enterprises ways to examine proposed uses before wider deployment.

The review should carry the same weight as other major business risks. “Enterprises must proactively diagnose AI risk with the same level of seriousness as financial or cybersecurity risk,” Palanichamy says. He recommends a dedicated, cross-functional AI safety and ethics board committee involving legal, cybersecurity, compliance, data engineering, and product development. Bringing these functions together allows approval to account for consequences across their areas of responsibility.

Approval must come before deployment, including when a supplier has provided safety documentation or guarantees. “No AI tool should be deployed without the appropriate sign off,” Palanichamy says. His recommendation gives the enterprise a clear decision point for assessing vendor evidence against its operating conditions. It also makes someone responsible for accepting the intended use.

Once a tool is approved, employees need to understand how to handle its output. Palanichamy recommends mandatory AI literacy training across the business, including how to recognize hallucinations and verify generated output “prior to acting on it”. Verification requires someone to examine an AI response before using it in a decision or task. This step matters especially when generated content enters an established workflow whose other inputs employees already trust.

Technical restrictions can help employees apply those rules. Palanichamy suggests considering filters that check incoming inputs for toxic content, along with strict system-prompt guardrails, constraints in the instructions that govern the model’s behavior. These controls shape how a tool can be used and what it can accept. Their suitability depends on the application, so they need to be assessed within the same approval process.

After approval, oversight should change as the application’s risk changes. Palanichamy recommends continuous risk categorization, meaning that the business keeps reassessing how much organizational exposure an AI application creates. Higher-risk uses may need more frequent auditing and stronger restrictions on sensitive information. Governance teams can then adjust review effort as the consequences of a use become clearer.

A client-facing medical or financial analysis bot illustrates how this judgment affects supervision. Palanichamy says such a system may warrant daily auditing and restrictions to protect sensitive information, “whereas an internal AI tool to help with summarizing public meeting transcripts may not need as much governance oversight,” he explains. The transcript tool still requires an accountable deployment decision. Its narrower information exposure supports a different level of ongoing scrutiny.

Key takeaways for decision-makers

• Approve deployments against business conditions: Enterprise approval needs to connect supplier evidence to the data, workflows, and compliance duties involved in a proposed use. Reviewers also need to establish whether the business can detect harmful behavior and intervene.
• Use shared standards to assess suppliers: The proposed frontier AI standards body could give buyers more consistent risk assessments and testing evidence. Procurement teams need requirements for current purchases while industry standards develop.
• Assign responsibility for AI failures: Deployment owners need to assess data leakage, system access, unreliable output, and compliance exposure. Define provider responsibilities and internal response ownership before a system enters a business workflow.
• Require evidence for every model release: Procurement and technical reviewers need documentation covering capabilities, known failure modes, and testing history. Put model updates through change management to assess how new behavior affects approved uses.
• Match oversight to deployment risk: A cross-functional approval process should bring together legal, cybersecurity, compliance, data engineering, and product development. Set auditing frequency, information restrictions, and employee verification requirements according to each application’s risk, then reassess as its use changes.