Why single turn AI security tests are no longer enough

Multi-turn attacks expose fundamental AI security weaknesses

Most AI security testing today still focuses on a single prompt followed by a single response. That approach is no longer enough. Real attackers do not stop after one failed attempt. They adjust their questions, learn from the model’s responses, and continue the conversation until they find a path around the safeguards. If your testing does not reflect that behavior, you are measuring the wrong problem.

Cisco’s research makes this very clear. The company tested 15 proprietary flagship AI models using both single-turn and multi-turn attacks. The difference was significant. Across 6,986 multi-turn attacks, success rates ranged from 7.89% to 88.3%, depending on the model. Every model showed meaningful exposure. Even more important, the models did not rank the same way under single-turn and multi-turn testing. A model that appeared secure in traditional testing could perform much worse when an attacker adapted over several exchanges.

This changes how executives should think about AI risk. Security is no longer only about preventing a bad prompt. It is about understanding how an AI system behaves throughout an entire conversation. Every response creates new context, and that context can be used by an attacker to influence future responses. As AI agents become responsible for more business processes, these extended interactions become the standard operating environment rather than the exception.

Amy Chang, Head of AI Threat Intelligence and Security Research at Cisco, emphasized that organizations cannot understand where an AI application will fail unless they understand how different attack methods affect the underlying model. She explained that multi-turn conversations more accurately represent how people actually interact with AI systems, making them a much better indicator of operational security than isolated prompts.

Cisco has also expanded its evaluation efforts beyond traditional manual testing. Chang described an agentic framework where AI agents analyze deployment scenarios, generate relevant attacks, decide which attacks are worth pursuing, execute them, and evaluate the results. That level of automation increases testing speed, but her conclusion is straightforward. Strong security still depends on getting the fundamentals right. Advanced testing should strengthen good security practices, not replace them.

For business leaders, the implication is practical. AI security metrics based only on one-shot testing can create confidence that does not hold up in production. Procurement decisions, governance reviews, and deployment approvals should increasingly require evidence that systems have been evaluated under realistic, multi-step attack scenarios. That produces a much more accurate picture of operational risk.

Many enterprises remain underprepared for agent security

The market is moving quickly, but many organizations are still building AI security on incomplete foundations. AI agents are gaining access to enterprise systems, customer information, financial data, and business workflows. Yet many companies continue to rely primarily on the default security controls provided by AI vendors and cloud platforms.

VentureBeat’s June 2026 Pulse survey highlights this gap. Among 107 enterprise respondents, 54% reported either a confirmed AI agent security incident or a near miss. Specifically, 18% experienced a confirmed incident, while 36% stopped an attack before damage occurred. At the same time, only 32% assigned each AI agent its own managed and scoped identity, and just 30% isolated their highest-risk agents inside sandbox environments. Meanwhile, 82% said provider-native or hyperscaler controls remained their primary layer of protection.

These numbers suggest that AI adoption is advancing faster than security maturity. Organizations are deploying increasingly capable agents without consistently applying identity management, access controls, and workload isolation that have been standard practices in cybersecurity for years. The result is a larger attack surface combined with limited visibility into what each agent is allowed to do.

The industry has recognized this shift. Major cybersecurity companies are investing heavily in technologies that strengthen identity, authorization, and isolation. Palo Alto Networks completed its $25 billion acquisition of CyberArk in February 2026. CrowdStrike agreed in January 2026 to acquire SGNL for $740 million. Cisco also announced its intention to acquire Astrix Security in a deal reported at approximately $400 million. Although each acquisition has its own strategic goals, together they signal that identity and access management have become central to securing AI agents.

For executives, the message is clear. AI security is becoming an architectural decision rather than a product decision. Buying an AI platform with built-in security features is valuable, but it does not eliminate the need for enterprise identity management, permission controls, monitoring, and isolation. Those capabilities determine how much damage an attacker can do if an AI agent is compromised.

Organizations should also recognize that AI agents deserve the same governance standards applied to human users and critical applications. Every agent should have a clearly defined identity, narrowly scoped permissions, continuous monitoring, and well-defined operational boundaries. Those investments improve security today while creating a stronger foundation as AI systems become more autonomous and more deeply integrated into business operations.

Security must emphasize timeless fundamentals over complex defenses

As AI systems become more capable, there is a tendency to believe that security must become equally complicated. That is not what the evidence shows. Advanced attacks are becoming more sophisticated, but the most effective defenses still come from applying security fundamentals consistently across the organization.

Amy Chang, Head of AI Threat Intelligence and Security Research at Cisco, addressed this directly during the panel. Cisco has invested heavily in advanced security testing, including AI agents that can analyze deployment scenarios, generate attack strategies, execute those attacks, and evaluate the outcomes automatically. Even after developing these advanced capabilities, Chang’s conclusion remained straightforward. Organizations do not need increasingly creative defenses as much as they need to execute the basics well.

One of Cisco’s primary recommendations is its Integrated AI Security and Safety Framework. According to Chang, the framework maps the ways AI systems can be compromised throughout the entire AI lifecycle, from the underlying models and data sources to software supply chains and deployment. Rather than responding only after incidents occur, organizations should understand where risks exist before systems reach production.

This approach also changes how security teams should investigate incidents. Instead of simply fixing the immediate problem, teams should work backward to understand exactly how an attack succeeded. That analysis helps identify weaknesses across governance, identity management, model behavior, data access, and operational controls. Once those patterns become clear, organizations can apply targeted protections that reduce future risk.

For executives, this has important implications. AI security should not become a separate discipline disconnected from existing cybersecurity programs. The strongest organizations will integrate AI governance into established security frameworks, risk management processes, compliance programs, and operational controls. That creates consistency across the business instead of introducing another isolated security function.

Security investments should also be prioritized according to business impact rather than technical complexity. It is often more valuable to strengthen identity controls, access management, monitoring, and incident response than to pursue highly specialized defensive technologies that address only a narrow set of threats. As AI adoption accelerates, disciplined execution of proven security practices will continue to deliver the highest return.

Continuous multi-turn red teaming and monitoring are essential

Deploying an AI system is not the end of security testing. It is the point where continuous validation becomes necessary. Models evolve, business processes change, permissions are updated, and attackers constantly adapt their methods. Security must evolve at the same pace.

Heather Ceylan, Chief Information Security Officer at Box, explained that many organizations still rely on single-turn red teaming even though users interact with AI through ongoing conversations. To address this gap, Box has developed adversarial AI agents that repeatedly attempt to compromise production agents through multi-turn interactions. These automated attackers continuously refine their approach, creating a much more realistic evaluation of how systems perform under sustained pressure.

One lesson stood out from Box’s operational experience. The company introduced AI agents into its security operations center with human approval required for every action. As confidence in the agents grew, analysts shifted toward monitoring rather than approving each decision. Then a single mistake occurred. According to Ceylan, the trust that had been built over time disappeared immediately, and the team had to rebuild confidence from the beginning.

That experience reflects an important reality for executive leadership. Trust in AI must be earned continuously through reliable performance, transparent oversight, and measurable accountability. A successful deployment today does not guarantee reliable behavior tomorrow because models, data, and external dependencies continue to change.

Continuous monitoring addresses this challenge. Organizations should monitor agent behavior, evaluate outputs, review permission changes, and detect unusual activity throughout an AI system’s lifecycle. Even if humans are no longer involved in every decision, they should retain visibility into how agents operate and maintain the ability to intervene when necessary.

For boards and executive teams, this means shifting performance metrics beyond deployment speed or automation gains. AI governance should include ongoing security validation, operational monitoring, incident reporting, and periodic reassessment of model behavior. These practices provide early visibility into emerging risks before they develop into business disruptions.

The broader message is clear. Security testing should become an ongoing operational capability rather than a milestone completed before deployment. Organizations that continuously test and monitor their AI systems will be better positioned to adapt as threats evolve and as AI agents take on increasingly important business responsibilities.

Layered permission controls and runtime restrictions mitigate damage

No security system can guarantee that an AI agent will never be compromised. The better objective is to ensure that if something does go wrong, the impact is limited. That requires security controls that operate before, during, and after an agent performs a task.

Heather Ceylan, Chief Information Security Officer at Box, described the company’s approach as three complementary layers. The first is permissioning. An AI agent should never receive broader access than the person who initiated the request. Every action begins with tightly controlled authorization that limits what the agent can see and do.

The second layer is task isolation. Box creates an ephemeral sandbox environment for each agent task. These temporary environments help contain any compromise to a single execution rather than allowing it to spread across systems or persist after the task is complete. This significantly reduces operational risk, particularly for agents interacting with sensitive business data.

The third layer focuses on runtime execution controls. Instead of allowing agents unrestricted access to available tools, Box limits each agent to only the functions required for its assigned task. Ceylan provided a practical example. If an agent’s purpose is to summarize a document, a prompt injection attack instructing it to forward that document to an external attacker cannot succeed because sending emails is not an authorized tool available during that task. The agent simply does not have permission to perform that action.

Box also organizes agent actions into three categories based on business risk. Low-risk activities, such as reading or summarizing information, can proceed without human approval. Moderately sensitive actions may execute automatically but are logged and monitored for review. High-risk actions, including destructive operations such as mass file deletion, always require human authorization before execution.

This structured governance gives organizations flexibility while maintaining control. As AI capabilities evolve, activities may move between these categories, but the underlying framework remains consistent. That consistency is valuable because it allows security policies to evolve without requiring organizations to redesign their entire governance model.

For executives, this approach reinforces an important principle. AI agents should not receive broad, permanent access simply because they may eventually need it. Every permission should be limited to a clearly defined business purpose, granted only when required, and removed when the task is complete. This reduces unnecessary exposure while improving auditability and regulatory compliance.

The broader business benefit extends beyond security. Clearly defined permissions make it easier to understand how AI systems interact with enterprise data, simplify compliance reporting, and improve confidence among employees, customers, and regulators. As organizations deploy more autonomous agents, disciplined permission management becomes an operational requirement rather than an optional security enhancement.

Centralized AI security platforms strengthen overall protection

As organizations deploy dozens or even hundreds of AI agents, managing security individually for every application becomes increasingly difficult. Different development teams often implement controls in different ways, creating inconsistencies that increase operational risk. A centralized security platform provides a more scalable solution.

Rajesh Parekh, Vice President of AI and Machine Learning at Intuit, described the company’s answer to this challenge: GenOS, short for Generative AI Operating System. Rather than expecting every development team to build its own security controls, GenOS provides centralized capabilities for identity management, security, fraud detection, and risk management. Developers inherit these protections automatically, allowing them to focus on solving business problems while maintaining consistent security standards.

One of the most significant changes Intuit has made involves identity management. Earlier AI systems often inherited the permissions of the human user. Intuit has moved away from that model. Today, each AI agent carries its own identity with narrowly defined, auditable permissions tied to specific responsibilities.

Parekh explained that permissioning is not about giving AI unrestricted access. It is about defining exactly what authority an agent has for a particular task and ensuring every action can be traced and reviewed. The company is also exploring dynamic permission changes during active sessions so an agent’s authority can adjust as the work changes instead of remaining static throughout the interaction.

Intuit also uses experience gained from security testing to strengthen future deployments. When manual red-teaming exercises identify recurring vulnerabilities, those attack patterns are converted into automated tests within the GenOS platform. Future AI agents are evaluated against those scenarios automatically before deployment. This allows security knowledge to accumulate across the organization rather than remaining isolated within individual projects.

Runtime monitoring provides another layer of protection. GenOS continuously scans prompts and model responses for suspicious behavior. If potentially unsafe activity is detected, the platform can interrupt execution and escalate the decision to a human expert. This creates a practical balance between automation and oversight without requiring people to review every interaction manually.

Parekh also described Intuit’s broader vision as an AI-powered expert platform where users, AI agents, and human experts work together. In this model, human expertise is built directly into operational workflows instead of serving only as a final approval step. That allows organizations to automate routine work while maintaining appropriate governance over higher-risk decisions.

For executive leaders, centralized AI security offers several strategic advantages. It simplifies governance, creates consistent enforcement of security policies, reduces duplicated engineering effort, and improves visibility across the organization’s AI estate. It also makes compliance easier because security controls, audit logs, and policy updates can be managed centrally rather than separately for every application.

As AI adoption accelerates, organizations will need security platforms that can scale with growing numbers of autonomous agents. Standardizing identity, monitoring, testing, and policy enforcement across the enterprise will likely become one of the defining capabilities of mature AI governance.

AI-driven software development is redefining security review processes

AI is changing software development at a much faster pace than previous development tools. Code is being generated more quickly, development cycles are becoming shorter, and security teams can no longer rely on review processes designed for slower release schedules. That means security itself has to become more automated.

Heather Ceylan, Chief Information Security Officer at Box, was direct about this shift. She argued that traditional secure code reviews conducted manually by engineers and security architects are no longer sufficient for the speed at which AI-assisted development is moving. Organizations that continue relying primarily on those methods risk slowing innovation without significantly improving security.

Instead, Box is working toward a development lifecycle where AI agents participate throughout the software engineering process. These agents review design documents, apply security requirements before implementation begins, examine code for vulnerabilities, and identify issues continuously as software evolves. Rather than concentrating security at the end of development, security becomes part of every stage.

Ceylan also expressed optimism about the long-term potential of AI-generated software. She believes AI models will continue improving their ability to write secure code with fewer vulnerabilities. At the same time, she acknowledged that the industry is still some distance from reaching that level of reliability. Human oversight remains necessary, particularly for high-impact systems and complex business logic.

This reflects an important reality for executives. AI should not be viewed as a replacement for governance. It is a tool that can significantly improve productivity and strengthen security when deployed within well-defined operational frameworks. Organizations should continue validating AI-generated code, monitoring production systems, and maintaining clear accountability for security outcomes.

Despite the advances in AI-assisted development, Ceylan emphasized that one principle has not changed: least-privilege access. AI agents should receive only the permissions required for their assigned responsibilities. Granting broad permissions during initial deployment may accelerate development in the short term, but it creates security challenges that become increasingly difficult to correct as systems expand.

This advice extends beyond software development teams. Organizations should define access policies before AI applications are widely deployed, rather than attempting to reduce permissions after agents become deeply integrated into business operations. Strong governance established early is typically more effective and less disruptive than redesigning security controls later.

For business leaders, the objective is not simply faster software delivery. The objective is sustainable development that maintains quality, security, and compliance while benefiting from AI-driven productivity. The organizations that succeed will integrate automation with disciplined governance instead of treating them as separate priorities.

Expanding agent capabilities increase attack surfaces

As AI agents become more capable, they also become more attractive targets. Every new capability, data source, external application, or business process connected to an agent creates additional opportunities for attackers. Greater functionality brings greater responsibility for security.

Rajesh Parekh, Vice President of AI and Machine Learning at Intuit, explained that AI agents now possess skills that extend well beyond generating text. They interact with enterprise data, execute workflows, access software tools, and make decisions within business processes. Those capabilities improve productivity, but they also increase the number of ways an attacker can attempt to exploit the system.

According to Parekh, an agent’s skills can themselves become vulnerabilities if they are not carefully governed. Access to external tools, sensitive information, and connected services increases the potential impact of malicious prompts or compromised behavior. As more systems become interconnected, the consequences of a successful attack can extend well beyond a single application.

This is why continuous security testing has become increasingly important. Intuit conducts manual red-teaming exercises to identify emerging attack patterns and weaknesses. Once recurring vulnerabilities are identified, those findings are incorporated into the company’s GenOS platform as automated security tests. Every future agent developed on the platform benefits from protections based on previous security experience.

The company also performs runtime scanning of prompts and model responses. Suspicious behavior can be identified while an interaction is taking place, allowing the system to stop potentially harmful actions before they are completed. When necessary, questionable decisions are escalated to human experts for further review.

Parekh emphasized that security testing cannot be treated as a one-time exercise. AI systems continuously change through model updates, new integrations, evolving business requirements, and changing dependencies. Organizations must continually validate that existing security controls remain effective under these changing conditions.

For executive teams, this requires a shift in governance priorities. Security should be evaluated whenever an AI agent receives new capabilities, connects to additional enterprise systems, or gains access to more sensitive information. Expanding functionality should always include an assessment of the additional risk being introduced.

The broader lesson is that AI capability and AI risk grow together. Organizations that actively manage this relationship through continuous testing, automated security validation, runtime monitoring, and disciplined governance will be better positioned to expand AI adoption with confidence. Those that focus primarily on new capabilities without strengthening security controls will likely encounter increasing operational and regulatory challenges as their AI environments become more complex.

Current AI systems cannot reliably infer user intent, making deterministic controls essential

One of the biggest questions in AI security is whether models can accurately determine what a user is truly trying to accomplish. Today, the answer is still no. AI systems can recognize patterns and estimate probabilities, but they cannot consistently determine intent with the level of certainty required for enterprise security.

This issue became a major topic during the panel discussion. Heather Ceylan, Chief Information Security Officer at Box, explained that Box has greater confidence when its own AI agent operates inside environments the company controls. In those cases, the system understands the context of the user’s request because it manages the interaction from the beginning. That allows Box to apply guardrails and tool restrictions with much greater precision.

The situation changes when external AI agents interact with enterprise systems. Those requests often arrive with incomplete or unknown context. Organizations may not know why an action is being requested or whether the request is part of a legitimate business process or a malicious attempt to manipulate the system. This uncertainty creates a significant security challenge.

Amy Chang, Head of AI Threat Intelligence and Security Research at Cisco, explained that current AI models are not trained in a way that allows them to reliably infer user intent from prompts alone. Because of this limitation, organizations should not depend on AI to determine whether a request is safe. Instead, they should implement deterministic controls that make security decisions based on clearly defined rules.

These controls include tightly scoped permissions, restricted tool access, identity verification, policy enforcement, and continuous monitoring. Unlike probabilistic judgments made by AI models, deterministic controls produce predictable outcomes that can be audited, tested, and consistently enforced across the organization.

The discussion also highlighted a broader industry debate. Mastercard is developing an open-source framework designed to standardize and communicate intent across AI interactions, particularly for complex business-to-business transactions. At the same time, some endpoint security vendors have indicated they are focusing less on intent inference and more on probability-based detection for production environments. This reflects an industry that is still exploring different approaches to one of AI’s most difficult security problems.

For executives, the practical takeaway is straightforward. AI should assist security decisions, but it should not become the final authority for access control or policy enforcement. High-impact actions should continue to depend on explicit authorization rules, verified identities, and clearly defined business policies.

As AI models continue to improve, their ability to interpret context will likely become more sophisticated. Even then, organizations should be cautious about replacing deterministic controls with model-based judgment. Predictability, accountability, and auditability remain essential requirements for enterprise security, regulatory compliance, and operational resilience.

Organizations should test AI the way attackers actually attack

The discussion throughout the panel led to one consistent conclusion. AI security is not a problem that organizations solve once. It is a capability that must improve continuously as models, business processes, and attack techniques evolve.

One of the clearest lessons is that security testing should reflect real attacker behavior. Attackers rarely rely on a single prompt. They adapt their approach, observe responses, change tactics, and continue interacting until they discover weaknesses. Organizations that evaluate AI systems using only isolated prompts are unlikely to uncover many of the vulnerabilities that appear during extended conversations.

This perspective is reinforced by the experiences shared by Cisco, Box, and Intuit. Cisco demonstrated through research that multi-turn testing reveals vulnerabilities that traditional evaluations often miss. Box showed that continuous monitoring is necessary because trust can be lost quickly after a single operational failure. Intuit emphasized that security testing should become part of an ongoing learning process, where new vulnerabilities identified during red teaming are automatically incorporated into future testing.

VentureBeat’s June 2026 Pulse survey suggests that many organizations still have work to do. The survey found that 82% of enterprises rely primarily on provider-native or hyperscaler controls as their main layer of AI security. It also reported that 59% are evaluating or planning to acquire dedicated agent security tools over the following 12 months. These findings indicate growing recognition that default platform protections alone are unlikely to meet long-term enterprise security requirements.

For executive leadership, this is ultimately a governance issue rather than simply a technology issue. Security reviews should evaluate whether AI systems undergo continuous testing, whether realistic attack scenarios are included in validation exercises, and whether monitoring continues after deployment. Success should be measured by an organization’s ability to detect, respond to, and learn from evolving threats rather than by the absence of known vulnerabilities at a single point in time.

The organizations that build these capabilities early will be better prepared as AI agents take on broader operational responsibilities. Continuous validation, disciplined identity management, strong permission controls, and ongoing monitoring create a stronger foundation for responsible AI adoption than relying solely on built-in platform protections.

The pace of AI development will continue to increase. Security practices need to improve at the same pace. Organizations that continuously test their systems under realistic conditions, strengthen governance as AI capabilities expand, and treat security as an ongoing operational function will be in a much stronger position to deploy AI confidently at enterprise scale.

Final thoughts

Enterprise AI is entering a different phase. The conversation is no longer centered on whether organizations should adopt AI. It is about whether they can operate AI responsibly as agents gain greater autonomy, connect to more business systems, and make increasingly important decisions.

The message from Cisco, Box, and Intuit is remarkably consistent. AI security cannot depend on one-time testing, broad permissions, or default platform protections. It requires continuous validation, disciplined identity management, narrowly scoped access, runtime monitoring, and governance that evolves alongside the technology. These are not separate initiatives. Together, they form the foundation of a secure AI strategy.

Business leaders should also recognize that AI security is becoming a competitive capability. Organizations that can demonstrate strong governance, explain how their AI systems are controlled, and respond quickly to emerging threats will earn greater trust from customers, regulators, partners, and investors. That trust will become increasingly valuable as AI becomes embedded in core business operations.

The pace of AI innovation will continue to accelerate. Security programs must keep pace without slowing the business. That means investing in scalable processes, automating where appropriate, and treating AI security as a continuous operational discipline rather than a compliance exercise completed before deployment.

The companies that succeed will not necessarily be those deploying the most AI agents. They will be the ones building AI on a foundation of strong governance, measurable accountability, and continuous improvement. As AI becomes a permanent part of enterprise operations, those capabilities will increasingly determine how confidently organizations can innovate while managing risk.