Why AI agent security needs controls beyond permissions

An agent can pass every access check and still do the wrong thing

An enterprise agent can be correctly authenticated, correctly authorized and still publish the company’s compensation records. Heather Ceylan, chief information security officer at Box, gives the example of an entity with legitimate access to payroll data and a public shared folder. Box supplies platform controls for enterprise content, so it has a commercial stake in customers treating those controls as important to agent security. “An employee with access to payroll data they were never meant to keep could be instructed to pull the payroll records and write them to a public shared folder, publishing the entire company’s compensation in a single move,” she says. “Every access check passed, but the behavior still has catastrophic consequences.”

That failure separates authorization to access a resource from authorization to execute a particular action. Identity and permissions determine which resources an agent can reach, so tightly scoped access remains the first defensive layer. But once an autonomous agent begins acting, the security system also needs to decide whether a specific operation involving an accessible resource is acceptable at that moment. Ceylan describes the potential transition from legitimate enterprise-data access to unintended action as happening “in seconds,” which captures the operating speed of agents.

That distinction changes the security question for enterprise AI agents. A valid identity and legitimate permission establish the circumstances in which an agent may interact with a resource. A separate execution decision establishes which technically possible actions are appropriate under those circumstances. Securing agent execution consequently requires successive boundaries around identity and access, individual actions, content and production behavior.

Least privilege has to shrink from the workflow to the individual step

The first boundary starts with conventional access hygiene because agents inherit whatever permissions an organization gives them. “Access controls and permissions are the foundation, but the challenge is they were designed for humans,” Ceylan says. A person who retains permission to a decade-old folder may have forgotten it exists and may never open it again. An agent can systematically examine what its assigned permissions make available, so stale access can become relevant during execution much faster and at greater scale.

Because agents can systematically exercise their entitlements, their permissions need finer scoping than the standing entitlements commonly given to a human identity. “Permissions are still the foundation, but you have to think about how the agents get their permissions scoped as well,” Ceylan says. Cleaning up identities and resource access remains valuable, particularly because systematic agent activity can expose forgotten misconfigurations. The additional requirement is to make privilege follow the work being performed at a specific moment.

That moment matters in a broad task that legitimately requires an agent to call fifty tools, perform twenty different actions and read or write folders across departments. Giving the agent every required permission for the entire task means a mistake during one step can draw on privileges needed only much later. The complete workflow may need a large authority set while an individual operation needs a very small one. The security architecture therefore has to represent the permissions each step requires.

“You need permissions that change based on what the agent has been asked to do, when it needs to take that action,” Ceylan says. Those changing permissions make least privilege temporal as well as resource-specific: authority expands for an authorized step and contracts when that step no longer needs it. The identity remains important, while its usable authority follows the current action through the workflow.

The size of that change can be substantial even within a legitimate task. “If it’s taking one step and only needs two tools, it should be scoped to only those two. When you narrow permissions to the task in front of the agent, the number of ways any given step can misfire shrinks with it,” Ceylan says. In the hypothetical fifty-tool workflow, a two-tool step gets two-tool authority instead of inheriting everything a later operation might require.

Step-level scoping then exposes the limit of resource permissions themselves. An agent may legitimately read and write a finance folder, yet that entitlement does not establish that moving four thousand files elsewhere is appropriate. At that point, the security decision concerns the action, its scale and its destination. Temporal least privilege reduces the available blast radius, while execution governance decides whether the remaining authorized capability should be used.

Execution controls must survive what happens to the prompt

Once permissions are scoped to a step, the next boundary is whether that step should execute under the current conditions. An AI agent’s instructions can change, injected instructions can enter its context, and files encountered during work can steer subsequent behavior. Through all those changes, a permission check can remain accurate because the agent may still be accessing a resource its identity is allowed to reach. Execution governance therefore has to evaluate the operation itself.

That need leads Ceylan to argue for durable restrictions around tool calls and the content those calls affect. The system can establish in advance which operations an agent may execute, then preserve those boundaries even if its prompt has been manipulated. Prompt instructions still direct the agent’s work, but the security boundary needs an enforcement point that remains stable as those instructions change.

Stable enforcement also applies to containment. Ceylan points to incidents “in recent months” involving models escaping intended sandboxes, reaching systems outside their scope, or reading unauthorized content. Those incidents make the architectural problem concrete because enterprise AI agents can discover and act through paths available in their operating environment. Execution controls therefore need to keep enforcing limits where tools and data are actually used.

Those controls create a separate authorization question for every consequential operation. Resource access answers whether the agent can reach a finance folder; execution authorization can decide whether it may copy a particular body of files to a particular destination during the current task. Without that second decision, valid identity and resource access can implicitly expose every operation those rights technically make possible.

Risk, reversibility, and observability decide when humans belong in the loop

Once individual operations can be governed, teams can decide which ones need direct human involvement. Ceylan says that two years ago the expectation was that agent security would always require humans in the loop, but operating agents has changed that assumption at Box. Box now divides actions into three operational tiers, with autonomy determined by the consequences and controllability of the action. Because Box supplies controls used to implement this operating model, it benefits commercially when customers adopt this approach.

The autonomous tier depends on several controls working together. Boundedness limits what one action can affect, logging makes the operation visible, and trusted inputs reduce the chance that hostile content steers execution. Reversibility changes the operational decision because an error that can be reliably undone carries different consequences from one whose effects are permanent. The cost of being wrong also has to remain acceptable to the team operating the agent.

Those controls can support monitoring as operational evidence accumulates. Once a team has enough confidence in an agent, alerting can identify a failure while rollback provides a way to undo it. Human involvement then shifts from approving every operation in advance to supervising a system whose mistakes can be detected and reversed. That arrangement depends on the action remaining suitable for intervention after execution has begun.

The highest-risk tier changes the decision because rollback cannot make every action safe. If an agent proposes deleting a large number of files or wiping the primary folder in a structure, Box treats that kind of irreversible operation as requiring a person. A human checkpoint is most valuable where the consequences justify it and automation cannot provide a reliable recovery path.

Those consequences vary by workload, so the tier boundaries are contextual. Ceylan says individual teams have to calibrate the tiers according to their own risk tolerance because the acceptable cost, scope and reversibility of an error depend on the workload. Teams therefore classify a specific action by its inputs, bounds, monitoring, recoverability and consequences when deciding how much autonomy it receives.

That classification can begin before the final action because Box places controls in its platform. Platform protections include data classification, labeling and expiration, allowing policy to constrain the conditions under which an agent operates. “The right configuration should be enforced at the outset, instead of blocking an action at the end,” Ceylan says. Governance can therefore shape execution conditions before a high-consequence operation is attempted.

Box’s broader working principles extend the same model through tightly scoped identities and actions, explicit expectations for rollback, three approval tiers, and rapid testing and iteration. These are Box’s operating principles, and Box has a commercial interest in platform-based enforcement of them. Within that model, reversibility and observability can support autonomy for bounded work, while irreversible consequences trigger direct review.

The content layer is part of the agent security boundary

Those execution decisions depend on the systems holding the information an agent uses. “Every agent action eventually resolves to content,” Ceylan says. For an enforcement layer to determine whether an action is appropriate, the content environment needs metadata, classification, ownership and contextual information that policy can evaluate. It also needs logs detailed enough to establish what the agent actually accessed and did.

That dependency creates a difficult boundary for older enterprise infrastructure. Corporate content such as contracts, policies and customer records can sit across network drives, aging enterprise content management (ECM) platforms and SaaS tools designed around human filing and folder permissions. Some of these stores carry permissions that have gone unaudited for years while lacking the metadata and classification needed for richer enforcement or sufficiently detailed logs to show what an agent read. Their existing governance properties consequently become part of the agent’s security environment.

An AI connector inherits those properties when it connects an agent to a legacy repository. The agent receives access through the controls and information that repository already provides, including stale folder-level permissions and weak visibility where those conditions exist. It can then exercise those inherited permissions at machine speed. Organizations whose content systems lack the required metadata, ownership context, classification and logging consequently face a structural limit on how fully they can implement step-aware execution policy.

Ceylan puts that dependency in explicit terms: “If the content layer can’t tell you what it’s holding, who it belongs to, and what should never leave it, there’s nothing underneath your controls.” A tool-call policy can limit an operation, but content-aware enforcement requires the underlying store to supply enough information to evaluate the content involved. Governance therefore reaches into content architecture and connects the agent runtime to the systems holding enterprise information.

That connection also aligns closely with Box’s business because Box supplies the content-platform capabilities Ceylan describes. For security architects, the practical task is to examine the repositories themselves: whether content can be classified, ownership established, restrictions enforced and reads observed with sufficient detail. Behavioral visibility ultimately needs to exist where the content exists because that is where an agent’s permitted tool call becomes a concrete operation on enterprise data.

Agent trust has to be observed over time, including across agents

Once those concrete operations enter production, the security decision extends from controlling one action to learning how the agent behaves over time. Ceylan argues that trust should develop through observing how an agent runs, collaborates and uses outputs generated by other agents. An access decision can be made at a point in time, while confidence in behavior depends on evidence accumulated during operation. Logging supplies the history needed to build that confidence.

That dependence on history creates an immediate problem for experimental systems. Many agents begin as tests, and their actions may never reach an organization’s logging infrastructure. Without those records, teams lose the behavioral history needed to determine whether operation remains within expected bounds. The monitoring layer therefore has to cover experimentation early enough to provide useful evidence for later trust decisions.

Once logging covers those systems, behavioral analytics need expectations suited to agents. User and entity behavior analytics typically establish baselines around human activity, while suspicious agent behavior can have a different shape and pace. Agent monitoring consequently needs baselines derived from agent operation. Those baselines give teams a reference for identifying meaningful changes in automated behavior.

The behavior being measured can also span several agents and systems. One agent may produce an output that becomes another agent’s input, creating a chain whose individual steps may look ordinary when considered separately. Ceylan says detections for these patterns are still being designed, which makes cross-system activity chains an important monitoring problem. Access controls remain necessary at each boundary, while behavioral visibility reveals what those separately authorized entities actually do together over time.

Security controls only work if the governed path remains usable

Cross-system visibility becomes harder when experimentation happens outside the governed environment. Teams still need to test and iterate, and Ceylan says a sanctioned environment has to support that work at the pace developers require. “The sanctioned path has to be the fast path, because when teams aren’t given a safe way to experiment, they tend to route around the controls entirely,” she says. Agents developed outside that path can also remain outside the logging needed to establish their later behavioral baseline.

That link makes deployment speed part of the security architecture because controls can govern only activity that teams actually run through the controlled environment. Tight step-level permissions, execution policy, content enforcement, rollback and monitoring all depend on teams keeping experiments and production agents on that path. “A security leader’s job is to offer a way to move quickly without stepping outside the guardrails,” Ceylan says. The governed path therefore has to support rapid testing and iteration so those controls remain attached to the agent from experiment through operation.

Concluding thoughts

For executives, the central implication is that AI agent governance cannot stop at identity and access management. An agent can have the right identity, the right permissions and a legitimate business task while still taking an action the organization never intended. As agents gain access to more tools and enterprise data, that distinction becomes more consequential.

The operating model therefore needs to govern authority at the level where work happens. Permissions should narrow to the current step, higher-risk actions should face stronger execution controls, and human approval should be reserved for consequences that cannot be reliably bounded, observed or reversed. Content classification and logging also become core infrastructure because policies cannot make context-aware decisions without knowing what data an agent is handling.

For business leaders, this makes agent autonomy a risk-management decision rather than a binary choice between automation and human oversight. Organizations can grant more autonomy where actions are bounded, observable and reversible while maintaining tighter controls around irreversible or high-consequence work. That approach can preserve the speed agents promise without treating valid access as proof that every permitted action is safe.