February 26, 2020

How Adversarial Attacks Could Destabilize Military AI Systems

Artificial intelligence and robotic technologies with semi-autonomous learning, reasoning, and decision-making capabilities are increasingly being incorporated into defense, military, and security systems. Unsurprisingly, there is increasing concern about the stability and safety of these systems. In a different sector, runaway interactions between autonomous trading systems in financial markets have produced a series of stock market “flash crashes,” and as a result, those markets now have rules to prevent such interactions from having a significant impact.

Could the same kinds of unexpected interactions and feedback loops lead to similar instability with defense or security AIs?

Read the full article from IEEE Spectrum.

Learn more about the Artificial Intelligence and International Stability Project:

Artificial Intelligence and International Stability Project

Despite calls from prominent scientists to avoid militarizing AI, nation-states are already using AI and machine-learning tools for national security purposes. AI has the pote...
Read More

Reports
- April 22, 2025
Promethean Rivalry
Executive Summary Just as nuclear weapons revolutionized 20th-century geopolitics, artificial intelligence (AI) is primed to transform 21st-century power dynamics—with world l...

By Bill Drexel
Reports
- March 13, 2025
Safe and Effective
The promise of artificial intelligence (AI) and autonomy to change the character of war inches closer to reality...

By Josh Wallin
Reports
- June 11, 2024
Catalyzing Crisis
The arrival of ChatGPT in November 2022 initiated both great excitement and fear around the world about the potential and risks of artificial intelligence (AI). In response, s...

By Bill Drexel & Caleb Withers
Congressional Testimony
- October 19, 2023
Obstacles and Opportunities for Transformative Change
Watch:...

By Paul Scharre