April 27, 2022

The U.S. and China Need Ground Rules for AI Dangers

The threats are bigger, the stakes are higher, and the level of trust between the United States and China is lower today than it was in 2014, when experts from both countries first began discussing the risks posed by artificial intelligence (AI). At a time when about 9 in 10 U.S. adults consider China to be a “competitor” or “enemy,” calls for Washington and Beijing to cooperate on shared challenges routinely fall on deaf ears. But, as laboratories in both countries continue to unveil dramatic capabilities for AI systems, it is more important than ever that the United States and China take steps to mitigate existential threats posed by AI accidents.

As a technology, AI is profoundly fragile. Even with perfect information and ideal operating circumstances, machine learning systems break easily and perform in ways contrary to their intended function. Since 2017, the Global Partnership on AI has logged “more than 1,200 reports of intelligent systems causing safety, fairness, or other real-world problems,” from autonomous car accidents to racially biased hiring decisions. When the stakes are low, the risk of an AI accident can be tolerable—such as being presented with an uninteresting Netflix recommendation or suboptimal driving route. But in a high-pressure, low-information military environment, both the probability and consequences of AI accidents are bound to increase.

As a technology, AI is profoundly fragile.

Weapon systems put at high alert, for instance, could mistake a routine incident for an attack—and even automatically respond. Some of the Cold War’s most dangerous nuclear warning malfunctions were narrowly avoided because human judgment prevailed. For now, nuclear command and control systems in the United States and China still require that element of human decision-making—but, for instance, shipboard defense systems that might be involved in naval confrontations do not.

Neither side trusts the other on this issue. Over the past six months, I have spoken on a handful of occasions with retired Chinese military leaders about the risks involved with AI systems. They view the U.S. Defense Department’s AI ethics principles and broader approach to “responsible AI” as bad-faith efforts to skirt multilateral negotiations aimed at restricting the development of autonomous weapons. Meanwhile, U.S. observers don’t believe China is serious about those negotiations, given its extraordinarily narrow definition of lethal autonomous weapons systems. (China has called for a ban only on autonomous weapons that cannot be recalled once initiated and which kill with indiscriminate effect.) Both militaries are developing automated target recognition and fire control systems based on AI, and the last substantial discussion among the United Nations Group of Governmental Experts focused on these issues is set to conclude in mid-2022.

Read the full article from Foreign Policy.

  • Commentary
    • Foreign Policy
    • December 4, 2024
    Trump Must Rebalance America’s AI Strategy

    The disagreements about AI progress are so fundamental and held with such conviction that they have evoked comparisons to a “religious schism” among technologists....

    By Bill Drexel & Ruby Scanlon

  • Commentary
    • November 26, 2024
    Guidance for the 2025 AI Action Summit in Paris

    In September 2024, the French government, in collaboration with civil society partners, invited technical and policy experts to share their opinions on emerging technology iss...

    By Janet Egan, Michael Depp, Noah Greene & Caleb Withers

  • Commentary
    • Sharper
    • November 20, 2024
    Sharper: Trump 2.0

    Donald Trump's return to the White House is widely expected to reshape America's global priorities. With personnel choices and policy agendas that mark a significant break fro...

    By Charles Horn & Gwendolyn Nowaczyk

  • Podcast
    • November 18, 2024
    Team America

    Kate Kuzminski, Deputy Director of Studies, and the Director of the Military, Veterans, and Society (MVS) Program at CNAS, joins to discuss President-elect Donald Trump nomina...

    By Katherine L. Kuzminski

View All Reports View All Articles & Multimedia