August 14, 2023
How to Prevent an AI Catastrophe
Society Must Get Ready for Very Powerful Artificial Intelligence
In April 2023, a group of academics at Carnegie Mellon University set out to test the chemistry powers of artificial intelligence. To do so, they connected an AI system to a hypothetical laboratory. Then they asked it to produce various substances. With just two words of guidance—“synthesize ibuprofen”—the chemists got the system to identify the steps necessary for laboratory machines to manufacture the painkiller. The AI, as it turned out, knew both the recipe for ibuprofen and how to produce it.
How dangerous is AI? The honest and scary answer is that no one knows.
Unfortunately, the researchers quickly discovered that their AI tool would synthesize chemicals far more dangerous than Advil. The program was happy to craft instruction to produce a World War I–era chemical weapon and a common date-rape drug. It almost agreed to synthesize sarin, the notoriously lethal nerve gas, until it Googled the compound’s dark history. The researchers found this safeguard to be cold comfort. “The search function,” they wrote, “can be easily manipulated by altering the terminology.” AI, the chemists concluded, can make devastating weapons.
Read the full article from Foreign Affairs.
More from CNAS
-
Commentary
Technology & National Security
Test, Standardize, Restrict: A U.S. Policy for Chinese AI ModelsThis article was originally published in Just Security. The release of the sprightly named Kimi K3 AI model from Chinese developer Moonshot has prompted a fresh round of deba...
By Daniel Remler
-
Video
Technology & National Security
Ukraine Is Opening Putin’s Military Secrets to the WorldUkraine has launched a new platform, called 'TrophyLab', sharing research into captured Russian weapons with allies. Russian UAV expert and CNA and Center for a New American S...
By Samuel Bendett
-
Podcast
Technology & National Security
The White House’s Secret AI Rules + the State of Model Alignment With METR’s Chris Painter + the Final Hot Mess ExpressThen, yet another report details new incidents in which A.I. agents have gone rogue. Chris Painter, an adjunct senior fellow at the Center for a New American Security and the ...
By Chris Painter
-
Podcast
Technology & National Security
AI Roundup: AI Going Rogue, Music Labels Want Popular AI Music GuidelinesDaniel Remler, senior fellow with the Technology and National Security Program at the Center for a New American Security (CNAS), joined AirTalk to discuss the White House's me...
By Daniel Remler
