AI agent adoption does not always prove security gains

CISOs need two kinds of evidence. Adoption evidence shows whether practitioners choose to use agents. Performance evidence shows whether agents improve speed, quality, safety, or another defined outcome. Security leaders should govern and evaluate agents as practitioners incorporate them into their workflows.

The strongest AI signal is adoption

A deployment decision requires evidence that separates an agent’s contribution from other factors affecting performance. Security leaders can identify workflows where practitioners choose to incorporate AI agents and examine similar uses inside their organizations. Productivity claims require a separate comparison that measures the agent’s incremental effect.

AI agents among leading teams

Security executives can draw a narrow conclusion from adoption evidence: when high-ranking competitors integrate agents into their workflows, governance and internal evaluation become relevant for similar organizational use. Adoption alone cannot establish whether differences result from access, policy, skill, preference, or another factor. It also cannot establish the agents’ incremental effect on team results.

This distinction creates two executive questions. Are skilled practitioners choosing to work with agents? Do agents independently improve their results? Adoption evidence addresses the first; the second requires evidence that isolates the technology’s contribution.

Performance gains do not establish causation

Stronger measured competition outcomes over time do not identify their cause. Team composition, challenge design, competition conditions, AI use, or other changes can affect comparisons across years. Isolating AI’s contribution requires a design that separates its effect from those factors.

Open competitions can show voluntary use under competition conditions. Estimating incremental performance requires comparisons that control relevant differences between assisted and unassisted work. Security teams can make that distinction operational by comparing agent-assisted work with a defined baseline and measuring outcomes such as time, accuracy, and safety.

Teams should also record where an agent contributed directly and where human review changed or rejected its output. This creates evidence tied to the organization’s own tasks and operating conditions. Leaders can then base deployment decisions on measured effects rather than adoption alone.

The emerging capability is managing human-AI work

Agent use creates a management requirement. Organizations that permit agents in routine security work need rules for directing them, reviewing their suggestions, and approving actions that affect systems or data. Validation requires enough technical knowledge to judge whether an output fits the specific environment and threat conditions. Human review therefore becomes part of the operating model for agent-assisted security work.

CISOs can turn this requirement into policy decisions: which tasks may use agents, what systems and data agents may access, where human approval is required, and who is accountable for resulting actions. Access controls, human approval, technical validation, and measured comparisons with defined baselines provide concrete safeguards that can be evaluated within existing security operations.

Key highlights

Adoption is the strongest signal: AI agent use by skilled practitioners can identify workflows worth evaluating, but CISOs should measure agents against defined internal baselines before making productivity claims.
Leading teams make governance relevant: When top cyber teams adopt agents, security leaders should assess similar uses while avoiding assumptions that adoption explains stronger results.
Performance gains require controlled evidence: CISOs should compare agent-assisted and unassisted work on measures such as time, accuracy and safety, while tracking where human review changes or rejects agent output.
Human-AI work needs clear controls: Define agent access, permitted tasks, human approval requirements, technical validation and accountability before agents become routine parts of security operations.