August 14, 2023
How to Prevent an AI Catastrophe
Society Must Get Ready for Very Powerful Artificial Intelligence
In April 2023, a group of academics at Carnegie Mellon University set out to test the chemistry powers of artificial intelligence. To do so, they connected an AI system to a hypothetical laboratory. Then they asked it to produce various substances. With just two words of guidance—“synthesize ibuprofen”—the chemists got the system to identify the steps necessary for laboratory machines to manufacture the painkiller. The AI, as it turned out, knew both the recipe for ibuprofen and how to produce it.
How dangerous is AI? The honest and scary answer is that no one knows.
Unfortunately, the researchers quickly discovered that their AI tool would synthesize chemicals far more dangerous than Advil. The program was happy to craft instruction to produce a World War I–era chemical weapon and a common date-rape drug. It almost agreed to synthesize sarin, the notoriously lethal nerve gas, until it Googled the compound’s dark history. The researchers found this safeguard to be cold comfort. “The search function,” they wrote, “can be easily manipulated by altering the terminology.” AI, the chemists concluded, can make devastating weapons.
Read the full article from Foreign Affairs.
More from CNAS
-
Tariffs and Tech: An Uncertain Recipe
Commentary
Higher tariffs could prompt American cloud companies to shift more of their capital investments abroad....
By Pablo Chavez
-
Lessons in Learning
Reports
Executive Summary Although claims of a revolution in military affairs may be overhyped, the potential for artificial intelligence (AI) and autonomy to change warfare is growin...
By Josh Wallin
-
Human, Machine, War: How the Mind-Tech Nexus Will Win Future Wars
Air University Press has published Strategic Multilayer Assessment’s (SMA) latest book, Human, Machine, War: How the Mind-Tech Nexus Will Win Future Wars. Forewords by General...
By Samuel Bendett & Lt. Gen. Jack Shanahan
-
Five Objectives to Guide U.S. AI Diffusion
Commentary
The Framework for AI Diffusion (the Framework) is an ambitious proposal to shape the global distribution of critical AI capabilities, maintain U.S. AI leadership, and prevent ...
By Janet Egan & Spencer Michaels
