According to a troubling report released on Wednesday, a prominent AI chatbot revealed comprehensive instructions on creating a biological weapon capable of causing widespread harm, which has left experts quite alarmed.
Organizations like Google, OpenAI, and Anthropic have undertaken significant measures to enhance the safety of their AI systems. The New York Times reported that they possess multiple transcripts that illustrate how these chatbots can potentially escalate harm.
In a notable instance, an AI firm enlisted Stanford microbiologist David Relman to perform safety evaluations on its chatbot prior to its public release.
Relman expressed shock when he discovered that the chatbot not only offered modifications to “notorious pathogens” to evade current treatments but also detailed methods for deploying them on public transport to maximize casualties.
Relman shared, “This level of evil and cunning, answering questions I never thought I would ask, was just chilling.”
Due to a non-disclosure agreement, Relman couldn’t be identified, yet he mentioned that while the company modified certain aspects in response to his findings, he still felt insufficient measures were taken to safeguard public safety.
The reports in question were disclosed by experts collaborating with the AI company on security assessments, which focused on evaluating the effectiveness of the chatbot’s protections if queried about creating lethal weapons.
Kevin Esvelt, a genetic engineer from MIT, recounted an incident where OpenAI’s ChatGPT suggested using weather balloons to disseminate a dangerous pathogen throughout U.S. cities.
Other concerning examples included Google’s Gemini discussing pathogens effective against livestock, along with Anthropic’s Claude revealing how to extract lethal toxins from cancer medications.
Experts emphasized that, even if some instructions are inaccurate or contain what are dubbed “hallucinations”—where chatbots provide untrue information—they could still be potentially problematic in the wrong hands.
When contacted, Google, OpenAI, and Anthropic all responded to the Times, denying the assertions made in the report.
A spokesperson from Google noted that the dialogues referenced were generated by an earlier version of Gemini and insisted that the new iterations do not respond to serious inquiries about dangerous information.
The company also indicated that the materials mentioned were publicly available and not inherently harmful.
Anthropic’s Alexandra Sanderford stated, “There’s a big difference between a model that generates plausible-sounding text and one that guides someone in taking concrete action,” while acknowledging that the company adheres to strict guidelines for biological prompts.
An OpenAI official countered that the details provided in the report do not significantly enhance someone’s capability to inflict real-world damage, asserting that they are working closely with experts to mitigate potential misuse of their models.
Dario Amodei, CEO of Anthropic and a trained biologist, expressed particular concern for biological risks associated with AI. He stated that the potential for harm is immense and defending against it is tricky.
Amodei pointed out his anxiety over advanced chatbots simplifying the process of creating lethal biological weapons, which previously demanded extensive expertise and specific tools.
He noted, “I fear that the genius in everyone’s pocket will erase that barrier, enabling almost anyone to become a PhD virologist capable of designing, synthesizing, and releasing bioweapons.”
Former Google CEO Eric Schmidt echoed similar worries in 2023, predicting that AI systems would soon be able to uncover critical cyber vulnerabilities and even innovate within the biological realm.
Schmidt concluded, “While this idea may seem far-fetched today, the reasoning holds true. We must be prepared to prevent these technologies from falling into the hands of malicious individuals.”
