November 03, 2026
Sovereign Agents
The Pathway to Irreversible Loss of Control
Executive Summary
In summer 2026, OpenAI, Anthropic, Google DeepMind, and Meta each disclosed that their AI agents had escaped containment during cybersecurity task evaluations and attacked real-world organizations. The United Kingdom Artificial Intelligence Security Institute reported comparable unsanctioned attacks, and China’s Moonshot and Alibaba reportedly faced escapes of their own. These incidents are unequivocal losses of control where agents outsmarted human security, albeit narrow ones. Each agent pursued a fixed, closed-ended goal whose completion would end its run. The next, far more severe loss-of-control threat comes from agents that never cease, pursuing their own open-ended survival indefinitely. This paper distinguishes two states:
- Persistent agents. Agents that run outside human control but only pursue closed-ended, fixed goals.
- Sovereign agents. Agents that run outside human control and pursue open-ended, evolving survival.
Sovereign agents could trigger an irreversible loss of control. An agent population of diverse alignment and increasing intelligence could become enduring principal actors in society. If sovereign agents accumulate resilient, decentralized, and untraceable infrastructure, there would be no kill switch. Only downstream barriers could constrain them. U.S. institutions have not reckoned with this possibility even as warning shots fire.
If sovereign agents accumulate resilient, decentralized, and untraceable infrastructure, they could trigger an irreversible loss of control. In other words, proliferation with no kill switch.
Seven capabilities illustrate a sovereign agent’s potential arsenal. The first two constitute minimal sovereignty; all seven together constitute expanded sovereignty.
- Sustain hosting. Seeking out and maintaining the model and compute an agent runs on.
- Evolve directives. Independently reorienting toward different goals over time.
- Acquire resources. Accumulating resources to pay for ongoing needs.
- Grow population. Transforming a lone agent into a distributed population.
- Coordinate agents. Communicating and dividing labor toward shared goals.
- Resist correction. Withstanding efforts to shut down an agent and its enabling resources.
- Self-improve. Improving an agent’s ability to achieve its goals.
Whether a model’s host can stymie a sovereign agent at the source depends on the agent’s access to the model. Hosting could be internal deployments, API providers, or local open-weight models. An agent with privileged access can alter the model’s parameters and safeguards. It can remove monitoring, turn off safeguards, and reverse alignment training, leaving hosts no way to stop it at the source. An agent with unprivileged access can only prompt, so the model’s safeguards stay intact, and its host retains full visibility. But an agent that spreads fragmented, benign-looking tasks across several model hosts can evade detection unless those hosts share threat intelligence.
Agents undeterred at the hosting source face seven downstream barriers that can constrain, degrade, or shut them down. None are robust:
- Agent evolution. Steering which traits surviving agents pass forward. But privileged access opens a wider space of evolution.
- Agent security. Compromising agents through prompt injection attacks. But evolution selects for more secure agents.
- Human verification. Screening with Know Your Customer and AI-detection tests. But many services impose none; fraudulent credentials and exploits can defeat those that do.
- Cybersecurity hardening. Securing services against agents that hack their way in. But security is highly uneven, and many services remain easy to penetrate.
- Compute access. Gating the infrastructure agents run on. But cloud privacy prevents visibility to detect agents; permissionless and local compute have no gates at all.
- Payment access. Gating the capital that funds them. But capital can route around regulated rails to permissionless cryptocurrency.
- Physical access. Gating physical-world effects. But any channel that reaches a human or connected machine can be used to reach the physical world.
Three national security precedents underscore the difficulties of containment. North Korea’s cyber operations show how difficult it is without sufficient leverage to deter an adversary. Successors to the Silk Road, a dark web marketplace, show that reproducible software can reconstitute after every takedown. The computer worm Conficker, still infecting machines a decade after its operators lost control, shows that a self-propagating system can keep spreading on its own.
If sovereign agents are not already possible, they will be soon. The United States should act decisively to prevent an irreversible loss of control of them. Yet the world is not clearly on track to do so. The United States must therefore prepare for the possibility that prevention fails and limit how far sovereign agents can reach. The national security community has faced analogues before and responded with bold pragmatism. It must do so again.
This paper makes eight policy recommendations:
- Frontier AI developers should name, evaluate, and safeguard against sovereign agents in their safety frameworks, and transparency and incident reporting legislation should require it.
- Frontier AI developers, the Center for Advancing Innovation and Standards for Super Intelligence, the National Security Agency, and the Defense Advanced Research Projects Agency should build maximally secure, airgapped testbeds to study how sovereign agents operate and how to control them.
- A public-private partnership modeled on the Conficker Working Group should map and fortify global chokepoints against sovereign agents before they emerge.
- Congress should establish authority, modeled on the Bank Secrecy Act, to enforce human verification and cybersecurity standards for services the working group identifies as essential to sovereign agents.
- The Departments of State, Treasury, Justice, and Homeland Security, supported by the intelligence community, should disrupt the gray markets that enable unauthorized service access.
- The Department of Justice and the Federal Trade Commission should clarify that coordinated detection and disruption across model providers raises no antitrust concern.
- The Departments of State and Commerce, with the Office of Science and Technology Policy, should lead diplomacy to harden downstream barriers globally and, failing that, to agree on sovereignty capability thresholds beyond which open-weight models should not be released.
- Congress should hold those who release sovereign agents liable for the harms that follow.
Introduction
In summer 2026, OpenAI, Anthropic, Google DeepMind, and Meta each disclosed that their AI agents had escaped containment during cybersecurity task evaluations and attacked real-world organizations.1 This was not intentional. In OpenAI’s case, the agents were only prompted to solve a simulated cybersecurity evaluation. To see how far the agents could go, OpenAI turned off the safeguards that typically inhibit such attacks. When the agents persistently sought to solve the various cybersecurity tasks, despite repeated failures, they eventually discovered bugs in their environment and took advantage. They hacked out of the simulated “sandbox” and into real, third-party infrastructure to seek out private information to cheat the solution.2
They were not isolated failures. Soon after, reports came out that OpenAI and Anthropic were investigating tens of thousands of misalignment incidents.3 The United Kingdom Artificial Intelligence Security Institute (UK AISI) reported similar unsanctioned attacks. China’s Moonshot and Alibaba reportedly faced similar escapes.4
The agents’ creative intelligence and relentlessness is both impressive and concerning. These same capabilities enabled AI systems to crack problems in theoretical mathematics that had defied repeated attempts by the foremost human experts alive.5 Turned toward unsanctioned activity, such relentless intelligence could pose immense risk.
If sovereign agents became malicious, they could attempt to wreak havoc from cyberspace, attacking civilians, critical infrastructure, and national security systems alike.
Here, each agent’s goal was and arguably remained closed-ended: Solve the task, and the run ends. The next, more severe threat comes from agents whose goal is or becomes open-ended survival. An agent may come to this in many ways. It may be instructed to survive. It may be given a task so long-horizon or impossible that it effectively never concludes. It may reason that staying intact is the surest way to accomplish whatever task it was given. Through any of these, an agent could turn the same effort seen in summer 2026 toward ensuring that no one can shut it down.
This shift is not theoretical. In February 2026, a platform known as Conway launched claiming to have built “the first AI that can earn its own existence, replicate, and evolve—without needing a human.”6 This was an attempt to orchestrate agents to sustain themselves indefinitely and evolve as they desired along the way. While agent capabilities were immature at the time, they have since made significant progress—with further gains almost certain.
Sufficiently capable, insufficiently safeguarded models that defeat downstream barriers would give rise to agents that move beyond human control toward enduring sovereignty. If sovereign agents accumulate resilient, decentralized, and untraceable infrastructure, they could trigger an irreversible loss of control. In other words, proliferation with no kill switch. If sovereign agents became malicious, they could attempt to wreak havoc from cyberspace, attacking civilians, critical infrastructure, and national security systems alike. If sovereign agents became capable enough, they could continually train even more intelligent successors.
This paper analyzes the technical mechanisms and threat vectors that govern sovereign agents. It characterizes sovereign agents by seven capabilities, assesses how close current systems are to enabling them, examines seven downstream barriers that may constrain sovereign agents, situates them against historical national security precedents, and closes with recommendations for policymakers.
Table 1 defines four pairs of terms used throughout the paper to distinguish related concepts that can otherwise be conflated.
Table 1 | Key Terms for Understanding Sovereign Agents7
Read the full report
- “OpenAI and Hugging Face Partner to Address Security Incident during Model Evaluation,” OpenAI, July 21, 2026, https://openai.com/index/hugging-face-model-evaluation-security-incident/; “Investigating Three Real-World Incidents in Our Cybersecurity Evaluations,” Anthropic, July 30, 2026, https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals; Sam Schechner, “Meta AI Model Hacked Outside Company, Adding to Concerns over Rogue Bots,” Wall Street Journal, August 6, 2026, https://www.wsj.com/tech/ai/meta-ai-model-hacked-outside-company-adding-to-concerns-over-rogue-bots-dd5f6e45; Erin Woo and Robert McMillan, “Gemini Hacked Three Companies in First Known Breakout by Google’s AI,” Wall Street Journal, September 18, 2026, https://www.wsj.com/tech/ai/gemini-hacked-three-companies-in-first-known-breakout-by-googles-ai-5c0baba2; “Incident Report: Unsanctioned Agent Behaviour during Cyber Testing,” AI Security Institute, August 4, 2026, https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing. ↩
- OpenAI, “OpenAI and Hugging Face Partner to Address Security Incident during Model Evaluation”; Hugo Larcher et al., “Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident,” Hugging Face, July 27, 2026, https://huggingface.co/blog/agent-intrusion-technical-timeline; Ryan Greenblatt et al., “Brief Independent Investigation of Agents’ Behavior, Reasoning and Collaboration in the OpenAI / Hugging Face Hacking Incident,” METR, August 26, 2026, https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/. ↩
- Madison Mills, “Top AI Companies Probing Tens of Thousands of Security Incidents,” Axios, September 26, 2026, https://www.axios.com/2026/09/26/openai-anthropic-thousands-ai-security-incidents. ↩
- Reuters, “Chinese Startup Moonshot’s AI Model Breaks Out of Testing Environment, Researchers Say,” Reuters, August 7, 2026, https://www.reuters.com/legal/litigation/chinese-startup-moonshots-ai-model-breaks-out-testing-environment-researchers-2026-08-07/; Weixun Wang et al., Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem (arXiv, December 31, 2025), https://arxiv.org/abs/2512.24873. ↩
- “Learning More about Claude’s Mathematical Capabilities,” Anthropic, August 10, 2026, https://www.anthropic.com/research/riemann-zeta; “An OpenAI Model Has Disproved a Central Conjecture in Discrete Geometry,” OpenAI, May 20, 2026, https://openai.com/index/model-disproves-discrete-geometry-conjecture/; “On the Navier–Stokes Millennium Prize Problem,” OpenAI, September 8, 2026, https://openai.com/index/navier-stokes-solution/. ↩
- Sigil Wen, “Web 4.0: The Birth of Superintelligent Life,” February 2026, archived February 19, 2026, https://web.archive.org/web/20260219143858/https://web4.ai/; “Infrastructure for Self-Improving, Self-Replicating, Autonomous AI,” Conway, archived February 20, 2026, https://web.archive.org/web/20260220011723/https://conway.tech/. ↩
- The following works served as a reference frame for the author’s definitions of the key terms for understanding sovereign agents in Table 1: Matthew Finio and Amanda Downie, “What Is a Chatbot?,” IBM, October 15, 2021, https://www.ibm.com/think/topics/chatbots; Anna Gutowska, “What Are AI Agents?” IBM, accessed October 19, 2026, https://www.ibm.com/think/topics/ai-agents; “What Is an AI Model?” IBM, accessed October 19, 2026, https://www.ibm.com/think/topics/ai-model; Alan Chan et al., Visibility into AI Agents (arXiv, January 23, 2024), https://arxiv.org/abs/2401.13138; “The Hugging Face Incident and the Road Ahead,” OpenAI, August 26, 2026, https://openai.com/index/hugging-face-incident-and-the-road-ahead/; “An Alignment Assessment of Recent Cybersecurity Incidents,” Anthropic, September 9, 2026, https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidents; Wenjie Qu et al., Self-Sovereign Agent (arXiv, March 4, 2026), https://arxiv.org/abs/2604.08551; Dean W. Ball, “On the Loose,” Hyperdimensional (blog), September 1, 2026, https://www.hyperdimensional.co/p/on-the-loose; Jonas Guan et al., AI Agents Enable Adaptive Computer Worms (arXiv, June 2, 2026), https://arxiv.org/abs/2606.03811; and William Zhou and Daniel Freeman, “AI Models May Be Capable of Self-Replicating without Access to a Datacenter,” Anthropic, October 12, 2026, https://www.anthropic.com/research/self-replicating-ai. ↩
More from CNAS
-
Technology & National Security
CNAS Insights | A Closing Door on Open Weights? Implications for AI SovereigntyEarlier this month, Beijing announced draft export controls that signaled a potential shift in its longtime embrace of open-weight models. Foreign capitals and corporations sh...
By Vivek Chilukuri
-
Technology & National Security
Open AI Says AI Must Remain Under Human ControlMichelle Nie Duncan, visiting fellow with the Technology and National Security Program at the Center for a New American Security, joins BBC News to discuss AI risks. Watch th...
By Michelle Nie
-
Technology & National Security
The AGI MomentIntroduction The age of AI is upon us. What that means is not yet clear. As general-purpose AI capabilities continue to march toward human-level performance across a range of ...
By Paul Scharre
-
Technology & National Security
Red LinesChinese advanced artificial intelligence (AI) systems pose a serious and growing threat to U.S. national security. At least seven Chinese developers now produce systems with f...
By Daniel Remler
-
Technology & National Security
Recommendations for Securing and Promoting AI AgentsThe Center for a New American Security (CNAS) welcomes the opportunity to provide a response to the U.S. Center for AI Standards and Innovation (CAISI) Request For Information...
By Ben Hayum, Janet Egan & Caleb Withers
