August 14, 2023
How to Prevent an AI Catastrophe
Society Must Get Ready for Very Powerful Artificial Intelligence
In April 2023, a group of academics at Carnegie Mellon University set out to test the chemistry powers of artificial intelligence. To do so, they connected an AI system to a hypothetical laboratory. Then they asked it to produce various substances. With just two words of guidance—“synthesize ibuprofen”—the chemists got the system to identify the steps necessary for laboratory machines to manufacture the painkiller. The AI, as it turned out, knew both the recipe for ibuprofen and how to produce it.
How dangerous is AI? The honest and scary answer is that no one knows.
Unfortunately, the researchers quickly discovered that their AI tool would synthesize chemicals far more dangerous than Advil. The program was happy to craft instruction to produce a World War I–era chemical weapon and a common date-rape drug. It almost agreed to synthesize sarin, the notoriously lethal nerve gas, until it Googled the compound’s dark history. The researchers found this safeguard to be cold comfort. “The search function,” they wrote, “can be easily manipulated by altering the terminology.” AI, the chemists concluded, can make devastating weapons.
Read the full article from Foreign Affairs.
More from CNAS
-
Biotech Matters: Public-Private Coordination of Biotechnology
Reports
An appreciation of biotechnology’s great opportunities is, for many commentators, intimately joined with regret about a disconnect between the U.S. government and the private ...
By Richard Danzig
-
$6.6 billion TSMC deal in Arizona the latest in the CHIPS Act’s rollout
Podcast
“President Biden signed the CHIPS and Science Act just about 20 months ago, which in government time is yesterday. And they’ve hired 200 people,” said Vivek Chilukuri, a senio...
By Vivek Chilukuri
-
Response to NTIA Request for Comment: “Dual Use Foundation Artificial Intelligence Models with Widely Available Model Weights”
Commentary
In February 2024, the Department of Commerce’s National Telecommunications and Information Administration (NTIA) issued a Request for Comment (RFC) on the implications of “ope...
By Caleb Withers
-
Biotech Matters: Great Data Competition and Interoperability with Allies and Partners
Reports
Understanding and strengthening America’s biotechnology leadership requires exploring the limits of existing data relevant to the U.S. bioeconomy so that policymakers and the ...
By Michelle Holko
