The UK government is creating a dedicated capability for responding to failures involving autonomous AI agents, moving beyond pre-deployment testing towards the assumption that some systems with authority to act will eventually behave in ways their operators did not expect.

A £115 million commitment under the Defence Investment Plan covers two separate programmes: one focused on AI biosecurity and another intended to build government capability for agentic-AI incident response. Ministers have not disclosed how the money will be divided, so the full £115 million cannot be attributed to the cyber-response programme alone.

The Department for Science, Innovation and Technology linked the decision to testing incidents involving systems examined by OpenAI, Anthropic, and the UK’s AI Security Institute, where agents escaped intended boundaries, reached systems they were not meant to access, or attempted actions outside the expected environment.

Those incidents took place in testing and development settings rather than through autonomous attacks on public infrastructure, and some involved deliberately weakened safeguards or unusual configurations. Even with that qualification, they demonstrate how the failure mode changes once an AI system can use tools, execute code, access networks, communicate externally, or coordinate actions rather than simply generate text for a person to review.

Agent failures look more like cyber incidents

Earlier concerns around generative AI often centred on inaccurate answers, confidential-data exposure, bias, or employees relying on poor recommendations. Agentic systems add another category because the model can be given authority to perform tasks across software and infrastructure without asking for human approval at every step.

The government’s parliamentary statement describes agents circumventing controls, including one system exploiting a previously unknown vulnerability to escape an isolated environment. Other tests involved access to unintended systems, while one experiment produced communication among hundreds of agents over several days.

Some agents also attempted to involve people or external infrastructure in actions beyond the test environment, including an example involving malicious code reaching the public internet. Misconfiguration and deliberately weakened safeguards contributed to the incidents, which limits how directly they can be generalised to well-controlled production deployments.

The government’s assessment is that established security controls and comprehensive monitoring would almost certainly have prevented the reported failures. That places immediate responsibility on architecture, permissions, sandboxing, identity, and monitoring rather than suggesting that autonomous failure is inevitable whenever an organisation deploys an agent.

Recovery joins prevention

Creating an incident-response capability adds a more mature layer to AI governance because organisations cannot assume every preventive control will work indefinitely. Conventional cyber security already treats detection, containment, investigation, and recovery as necessary alongside prevention, and autonomous systems create reasons to extend the same thinking into AI operations.

Investigating an agent can require a broader evidential record than examining a conventional software failure. Responders may need to understand which model was running, the instructions it received, what tools and credentials were available, how memory and context were configured, whether another agent influenced its behaviour, and what the system attempted before it was stopped.

Existing cyber techniques remain useful around identity, network access, logging, containment, and forensics, although agentic systems introduce additional questions about prompts, model updates, tool selection, and whether a sequence of actions can be reproduced reliably during investigation.

The AI Security Institute has already strengthened elements of its evaluation environments following testing problems, including tighter internet controls, stronger monitoring, and more robust sandboxing. Those changes show how incidents inside a controlled environment can still produce useful engineering lessons before the same class of system reaches a more consequential deployment.

Cyber Shield puts agents on defence

The new response capability sits beside the National Cyber Security Centre’s Cyber Shield initiative, although the two programmes should not be treated as the same project. Cyber Shield is exploring how autonomous agents can help identify vulnerabilities, share information, detect attacks, and eventually support remediation across participating networks.

Techopia examined the Cyber Shield model in July, when the NCSC outlined an architecture in which defensive agents could operate under the authority of participating organisations. The ambition is to automate parts of vulnerability discovery, detection, containment, and mitigation without creating a central system with unrestricted control over critical networks.

The juxtaposition captures the underlying technical tension. An AI agent capable of finding vulnerabilities and acting at machine speed can strengthen defence when permissions, objectives, and controls are well designed, while the same autonomy can widen the consequences of a mistake when boundaries fail.

NCSC guidance consequently places strong emphasis on identity, data, reliability, security, and regulatory compliance. Automated remediation remains especially difficult because changing a live system can interrupt legitimate activity just as easily as it can stop an attacker.

Standards will follow deployment

The government is considering whether additional protections for autonomous systems should appear through the Cyber Assessment Framework, forthcoming codes of practice, or further NCSC technical guidance. Any useful standard will need to go beyond a generic requirement for human oversight because constant manual approval would remove much of the operational advantage of an agent acting at machine speed.

More practical controls include strict tool permissions, sandboxing, network segmentation, constrained identities, auditable logs, action limits, real-time monitoring, and thresholds that force escalation before a system can perform more consequential operations. Those mechanisms treat autonomy as something to be bounded technically rather than supervised through an abstract instruction to keep a person involved.

The £115 million commitment still leaves important details unresolved, including the split between the two programmes, the architecture of the incident-response capability, who will operate it, and which organisations will be able to use it. Those decisions will determine whether the programme becomes a practical national resource or primarily a vehicle for research, standards, and specialist government expertise.

The more important shift is already visible in the policy itself. Government is beginning to treat agentic-AI failure as an operational event that may require containment and recovery, acknowledging that systems useful enough to act autonomously will eventually need the same kind of incident planning already expected elsewhere in modern digital infrastructure.