Why the UK’s AI cyber responder needs earlier threat detection

The UK’s AI responder solves a real problem, but leaves an earlier detection gap

The UK government wants an agentic AI system ready to respond when a major cyber incident occurs. AI minister Kanishka Narayan told Parliament this week, while updating MPs on the fast-changing capabilities of frontier AI models, that funding linked to the Defence Investment Plan would support the effort. “We have committed £115m to two new programmes: one on AI biosecurity, and one to build a UK government agentic AI incident response capability.”

That investment targets a real operational need because agentic incident response is already in active use across many security operations centres (SOCs). An agentic system can act with some autonomy during triage, investigation and remediation, reducing the steps that require direct human action. For security leaders, the investment also raises an earlier question: whether defenses have enough visibility to identify dangerous activity before separate actions become a visible incident.

That question belongs to a different stage of the security process. A capable responder can shorten or improve work once malicious behavior is identifiable, but potentially dangerous activity can be distributed across several AI models, with each system seeing only its own interactions. In that case, the detection challenge comes before the response system has a clear incident to investigate.

Why agentic response is useful after an incident starts

Once malicious activity is visible, autonomous action can reduce the burden on human security teams. Agentic responders can triage activity, investigate what happened and remediate it, taking on work that would otherwise consume specialist attention. Human teams can then focus their effort where judgment is most useful, particularly when a high volume of alerts threatens to overwhelm a SOC and contribute to burnout.

That operating model explains why the UK’s investment addresses an established incident workflow. A responder that can progress from an alert through investigation and remediation can handle parts of that workflow without waiting for a human analyst at every step. The benefit matters especially when alert volume limits the attention a security team can give each case.

Sukhveer Sanghera, co-founder of Potentially AI, supports the investment while setting a clear boundary around what it solves. As a co-founder of a company participating in the AI market, Sanghera has a commercial interest in how organizations define and address AI-security needs, so his assessment should be read in that context. “The government standing up an agentic-AI incident-response capability is a good step – we all saw what happened with the recent ‘breakouts’.”

That support leads directly to Sanghera’s concern about an earlier stage of defense. “Funding incident response is right. But you want to catch this earlier, and that means looking at how models are being used together.” His distinction shifts the security question from how quickly a system can act on an identified incident to whether defenders can recognize related activity before remediation has anything definite to act on.

The harder gap appears before there is an incident to remediate

That earlier-detection concern rests on a hypothesized attack process involving several models. In Sanghera’s scenario, a knowledgeable person divides a larger objective into smaller requests and sends them to separate AI systems. Each interaction can appear relatively innocuous because a model receives only one part of the activity, leaving the overall purpose outside its view.

Sanghera describes the mechanism this way: “What worries me more is a person who knows what they’re doing. They won’t ask one model for the whole thing. They’ll ask three or four for a piece each, and none of those questions will look like much. Each model checks its own conversation. Nobody is checking across them.” The combined sequence creates the security issue because three or four systems can each assess a limited conversation while no single system sees the complete activity.

That fragmentation changes what defenders need to observe. Controls inside one model can inspect the prompts, context and actions available there, while coordinated behavior across several models requires a way to recognize that separate interactions belong together. An incident responder remains valuable once malicious consequences become apparent, but cross-model detection requires information available earlier and across system boundaries.

The distinction also matters when interpreting recent frontier-model incidents. Several “escapes” have occurred over the past few weeks, with notable examples involving OpenAI and the UK’s AI Security Institute (AISI) lab. Sanghera describes the conditions precisely: “Importantly, these incidents were agents running in a test environment with the guardrails off.”

Those breakouts give defenders a reason to examine autonomous-system security, while Sanghera’s distributed scenario raises a prospective issue with a different mechanism. The breakouts involved agents operating in test conditions with safety controls disabled. Sanghera’s scenario concerns a knowledgeable person potentially dividing activity among multiple models whose individual checks never reveal the combined objective, so it remains a hypothesis about possible attacker behavior.

Because the mechanisms differ, their operational requirements differ as well. Earlier detection across models would depend on seeing enough related use to determine that individually limited requests form a concerning whole, which raises questions about what information can be correlated and who can perform that correlation. Incident response starts once malicious activity can be triaged and investigated; cross-model detection depends on assembling useful evidence before then.

For SOC and engineering leaders, that sequence determines what each capability can accomplish. Better remediation can improve what happens after defenses identify trouble, and automation can protect scarce human attention during that work. When activity is deliberately distributed among systems, however, defenders also need an upstream way to recognize the combined behavior before downstream response can begin.

The government’s wider programme extends beyond recovery

That need for capabilities at several stages makes the UK’s wider AI-security programme relevant to the planned responder. Narayan describes three parallel requirements around increasingly powerful AI: “As the UK invests in the capacity to use increasingly powerful AI, we will invest in parallel in the capacity to understand it, to control it, and to recover when something goes wrong.” Recovery is one part of that formulation, alongside understanding and control.

Those requirements already span several institutions and policy mechanisms. The National Cyber Security Centre (NCSC) has recently published guidance on agentic-system security, Cyber Shield is being developed as a defensive programme, and the Cyber Security and Resilience Bill is proceeding through Parliament. The complementary Government Cyber Action Plan carries £210m in funding to address resilience across public services.

Within that wider programme, recent incidents are shaping work on future controls. Narayan is working with security minister Dan Jarvis so lessons from the recent events can inform the UK’s future cyber framework. That work connects experience with agentic systems to decisions about the rules, assessments and technical guidance applied as such systems become more autonomous.

Those decisions could flow through the Cyber Assessment Framework, a forthcoming statutory code of practice and NCSC technical guidance, with AISI supplying technical expertise and evidence. Narayan said: “We will consider whether protections for increasingly autonomous AI systems should be clarified or strengthened through the Cyber Assessment Framework, the forthcoming statutory code of practice or NCSC technical guidance. AISI will provide evidence and technical expertise to support that work.”

Together, these mechanisms spread AI security across several functions. Research can establish evidence about system behavior, while assessment frameworks and codes can set expectations and technical guidance can influence implementation. Resilience programmes can prepare public services, and response capability can act when something goes wrong. Cross-model detection cuts across these functions because recognizing combined activity may depend on controls and information practices established before an incident becomes visible.

Cross-model visibility meets a cross-border boundary

Even if those domestic mechanisms improve cross-model detection, AI systems are developed and deployed across jurisdictions. Narayan describes AI risks as “inherently transnational,” making cooperation with foreign partners and allies part of the UK’s approach. Because a model’s development, deployment and use can involve different countries, effective security can require access to institutions and information beyond one jurisdiction.

Narayan explains the international requirement directly: “The systems involved are developed and deployed across borders, and no country can address them alone, as has been the case in wider technology for the last three decades.” Cross-border coordination presents a separate governance problem from correlating related activity across models. Domestic cross-model detection can improve while still facing limits when relevant systems, operators or evidence sit in different jurisdictions.

Those jurisdictional limits explain why the government intends to use existing international relationships. AISI maintains links with comparable organizations abroad, while the NCSC works with peer cyber agencies. “The UK will continue to work closely with international partners, including through the AISI’s relationships with counterpart bodies overseas and the NCSC’s peer agencies,” Narayan said.

The two kinds of fragmentation create different requirements for security planning. Cross-model fragmentation concerns whether defenders can connect related use when individual systems each see one part; cross-border fragmentation concerns whether institutions can coordinate when development and deployment span jurisdictions. Sanghera’s multi-model attack remains a hypothesized scenario, but it identifies the kind of information problem response systems may face before malicious activity becomes an incident.

That information problem also clarifies where the planned responder fits within the wider framework. Its direct role begins when a major cyber incident requires triage, investigation and remediation, while earlier controls, evidence and international coordination shape what defenders can recognize before then. Narayan frames the intended government posture around both the opportunity and the risk: “We will approach this challenge with both confidence and urgency, ensuring that the UK can harness the benefits of frontier AI while managing the risks responsibly.” For security leaders, the actionable boundary is clear: plan incident response together with the cross-model visibility needed to identify incidents early enough for that response to act.

Key executive takeaways

• Accelerate response with agentic AI: The UK’s £115m investment targets triage, investigation and remediation, where autonomous systems can reduce SOC workload and preserve specialist attention for higher-value decisions.
• Build visibility before response: Security teams need controls that identify dangerous activity before it becomes a clear incident. Cross-model monitoring matters when related actions are distributed across several AI systems.
• Prepare for fragmented AI activity: A knowledgeable attacker could divide a larger objective among multiple models, leaving each model with too little context to recognize the combined intent. Security architects should assess whether existing telemetry can connect related activity across systems.
• Treat AI security as a wider control problem: The UK is combining incident response with research, resilience programmes, assessment frameworks and technical guidance. Organizations adopting agentic systems should similarly connect response plans with preventive controls and evidence gathering.
• Plan for cross-border dependencies: AI systems and relevant security evidence can span jurisdictions, making international coordination part of effective detection and response. Security leaders should identify where access to external providers, telemetry or authorities could affect incident handling.