August 14, 2023
How to Prevent an AI Catastrophe
Society Must Get Ready for Very Powerful Artificial Intelligence
In April 2023, a group of academics at Carnegie Mellon University set out to test the chemistry powers of artificial intelligence. To do so, they connected an AI system to a hypothetical laboratory. Then they asked it to produce various substances. With just two words of guidance—“synthesize ibuprofen”—the chemists got the system to identify the steps necessary for laboratory machines to manufacture the painkiller. The AI, as it turned out, knew both the recipe for ibuprofen and how to produce it.
How dangerous is AI? The honest and scary answer is that no one knows.
Unfortunately, the researchers quickly discovered that their AI tool would synthesize chemicals far more dangerous than Advil. The program was happy to craft instruction to produce a World War I–era chemical weapon and a common date-rape drug. It almost agreed to synthesize sarin, the notoriously lethal nerve gas, until it Googled the compound’s dark history. The researchers found this safeguard to be cold comfort. “The search function,” they wrote, “can be easily manipulated by altering the terminology.” AI, the chemists concluded, can make devastating weapons.
Read the full article from Foreign Affairs.
More from CNAS
-
Commentary
Indo-Pacific Security / Technology & National Security
What It Takes to Stop the Next Salt TyphoonThis article was originally published on Just Security.Nearly a year after U.S. agencies identified one of the most severe cyber breaches of U.S. telecommunications companies,...
By Morgan Peirce
-
Commentary
Energy, Economics & Security / Technology & National Security
Beyond Bans: Expanding the Policy Options for Tech-Security ThreatsStuck between a rock (the fact that banning all Chinese tech that poses a risk is expensive and impractical) and a hard place (the fact that many existing mitigation proposals...
By Geoffrey Gertz
-
Reports
Indo-Pacific Security / Technology & National Security
Cyber Crossroads in the Indo-PacificThe Indo-Pacific faces a cyber crossroads. Down one path lies deeper military, intelligence, and economic ties between Washington and its key allies and partners in this strat...
By Vivek Chilukuri, Lisa Curtis, Janet Egan, Morgan Peirce, Elizabeth Whatcott & Nathaniel Schochet
-
Commentary
Technology & National Security
Securing America’s AI Future: Federal Research and Development PrioritiesOn April 29, 2025, the White House Office of Science and Technology Policy (OSTP) issued a Request for Information on the Development of a 2025 National Artificial Intelligenc...
By Caleb Withers & Spencer Michaels
