August 14, 2023
How to Prevent an AI Catastrophe
Society Must Get Ready for Very Powerful Artificial Intelligence
In April 2023, a group of academics at Carnegie Mellon University set out to test the chemistry powers of artificial intelligence. To do so, they connected an AI system to a hypothetical laboratory. Then they asked it to produce various substances. With just two words of guidance—“synthesize ibuprofen”—the chemists got the system to identify the steps necessary for laboratory machines to manufacture the painkiller. The AI, as it turned out, knew both the recipe for ibuprofen and how to produce it.
How dangerous is AI? The honest and scary answer is that no one knows.
Unfortunately, the researchers quickly discovered that their AI tool would synthesize chemicals far more dangerous than Advil. The program was happy to craft instruction to produce a World War I–era chemical weapon and a common date-rape drug. It almost agreed to synthesize sarin, the notoriously lethal nerve gas, until it Googled the compound’s dark history. The researchers found this safeguard to be cold comfort. “The search function,” they wrote, “can be easily manipulated by altering the terminology.” AI, the chemists concluded, can make devastating weapons.
Read the full article from Foreign Affairs.
More from CNAS
-
Commentary
Defense / Indo-Pacific Security / Technology & National Security
To Compete with China on Military AI, U.S. Should Set the StandardsThe United States has an opportunity to lead in global norms and standards for military AI at a critical moment, when the foundations laid today could shape how militaries use...
By Jacob Stokes, Paul Scharre & Josh Wallin
-
Video
Defense / Energy, Economics & Security / Technology & National Security
The Outlook CEO Perspectives on Risk, Resilience and ReturnsJoin David Schwimmer and Richard Fontaine, CEO of the Center for New American Security, as they explore the current national security landscape and its impacts on global econo...
By Richard Fontaine
-
Commentary
Technology & National Security
The Trump-Xi Truce Shows Who Really Holds the LeverageThis article was originally published in The Australian Financial Review. Since President Trump returned to office, the U.S.-China relationship has lurched between confrontat...
By Ruby Scanlon
-
Commentary
Technology & National Security
NOTUS Perspectives: AI Requires Massive Allocations of Energy. Will Other Sectors Suffer?On the current trajectory, the government may need to prioritize energy for AI over other uses to ensure the United States remains the global leader in advanced AI....
By Janet Egan
