August 14, 2023
How to Prevent an AI Catastrophe
Society Must Get Ready for Very Powerful Artificial Intelligence
In April 2023, a group of academics at Carnegie Mellon University set out to test the chemistry powers of artificial intelligence. To do so, they connected an AI system to a hypothetical laboratory. Then they asked it to produce various substances. With just two words of guidance—“synthesize ibuprofen”—the chemists got the system to identify the steps necessary for laboratory machines to manufacture the painkiller. The AI, as it turned out, knew both the recipe for ibuprofen and how to produce it.
How dangerous is AI? The honest and scary answer is that no one knows.
Unfortunately, the researchers quickly discovered that their AI tool would synthesize chemicals far more dangerous than Advil. The program was happy to craft instruction to produce a World War I–era chemical weapon and a common date-rape drug. It almost agreed to synthesize sarin, the notoriously lethal nerve gas, until it Googled the compound’s dark history. The researchers found this safeguard to be cold comfort. “The search function,” they wrote, “can be easily manipulated by altering the terminology.” AI, the chemists concluded, can make devastating weapons.
Read the full article from Foreign Affairs.
More from CNAS
-
Commentary
Technology & National Security
CNAS Insights | The Export Control Loophole Fueling China's Chip ProductionThis week, Reuters reported that China has apparently built a prototype of an extreme ultraviolet lithography (EUV) system, a highly intricate machine used to produce cutting-...
By Michelle Nie, Autumn Dorsey & Janet Egan
-
Video
Technology & National Security
Paul Scharre on How AI Could Transform the Nature of WarPaul Scharre, CNAS Executive Vice President, joins host Luisa Rodriguez to explain why we are hurtling toward a “battlefield singularity” — a tipping point where AI increasing...
By Paul Scharre
-
Commentary
Energy, Economics & Security / Technology & National Security
Recommendations for Promoting American AI AbroadStrategic Context and Program Objectives The American AI Exports Program is an ambitious and essential proposal to expand the reach of American AI technologies in foreign mar...
By Janet Egan, Geoffrey Gertz, Daniel Remler & Ruby Scanlon
-
Podcast
Technology & National Security
Defense One Radio, Ep. 200: Paul Scharre Explains the Global AI Arms RacePaul Scharre, executive vice president at the Center for a New American Security, joined Defense One to discuss science and technology. The former Army Ranger explored how AI ...
By Paul Scharre
