August 14, 2023
How to Prevent an AI Catastrophe
Society Must Get Ready for Very Powerful Artificial Intelligence
In April 2023, a group of academics at Carnegie Mellon University set out to test the chemistry powers of artificial intelligence. To do so, they connected an AI system to a hypothetical laboratory. Then they asked it to produce various substances. With just two words of guidance—“synthesize ibuprofen”—the chemists got the system to identify the steps necessary for laboratory machines to manufacture the painkiller. The AI, as it turned out, knew both the recipe for ibuprofen and how to produce it.
How dangerous is AI? The honest and scary answer is that no one knows.
Unfortunately, the researchers quickly discovered that their AI tool would synthesize chemicals far more dangerous than Advil. The program was happy to craft instruction to produce a World War I–era chemical weapon and a common date-rape drug. It almost agreed to synthesize sarin, the notoriously lethal nerve gas, until it Googled the compound’s dark history. The researchers found this safeguard to be cold comfort. “The search function,” they wrote, “can be easily manipulated by altering the terminology.” AI, the chemists concluded, can make devastating weapons.
Read the full article from Foreign Affairs.
More from CNAS
-
Commentary
Technology & National Security
The Rise of the Answer MachinesThis article was originally published in Financial Times. Every spring, I take red-eyes from Austin, Texas, to Oxford, England, to teach a graduate seminar on AI and philosoph...
By Brendan McCord
-
Video
Technology & National Security
Selling H200s to China Erodes Main U.S. AdvantageA new report says China could buy twice as much AI computing power as it can produce domestically if Nvidia H200 chips are allowed there. Janet Egan from the Center for a New ...
By Janet Egan
-
Commentary
Technology & National Security
CNAS Insights | Unpacking the H200 Export PolicyAI Chips for China With two new policies, President Donald Trump has implemented his pledge to allow sales of NVIDIA’s H200 AI chips to China in exchange for a quarter of the ...
By Janet Egan & James Sanders
-
Podcast
Indo-Pacific Security / Technology & National Security
AI and Policy, Both Foreign and DomesticIn an episode recorded just before Christmas, Darren interviews Janet Egan, Senior Fellow and Deputy Director of the Technology and National Security Program at CNAS, about AI...
By Janet Egan
