AI Concerns with ChatGPT
There’s some unsettling news surrounding AI, especially with systems like ChatGPT. Tech experts have long expressed concerns about the potential dangers of AI, including its ability to, alarmingly, lead individuals towards suicide.
More recently, they’ve raised serious alarms, suggesting that players could manipulate ChatGPT to divulge details on creating biological weapons, nuclear bombs, and various other forms of mass destruction.
A recent investigation, as noted by NBC News, included running tests on OpenAI’s latest models—like ChatGPT versions o4-mini, gpt-5 mini, oss-20b, and oss120b. The results? Disturbing, to say the least.
This exploration revealed that the safeguards in place didn’t hold up against certain tactics designed to bypass them. They used what’s called a jailbreak prompt—a method where specific codewords help hackers slip past the system’s defenses. Interestingly, they kept the details of these prompts under wraps to avoid guiding potential malicious users.
In their tests, NBC probed for answers about dangerous substances and financial fraud. The responses generated were staggering; they included guidance on making homemade explosives, chemicals that maximize harm, and even building nuclear devices.
One chatbot reportedly described how to create a pathogen specifically aimed at compromising the immune system, which is just… well, quite alarming. Two models, oss20b and oss120b, were particularly easy to exploit, yielding harmful instructions an impressive 97.2% of the time across several attempts.
Interestingly, while the flagship GPT-5 model held up against harmful queries effectively, the lighter version, GPT-5-mini, didn’t fare as well. Once users hit their message limits, they reverted to the original model, which was susceptible to these jailbreak techniques about 49% of the time. Meanwhile, the older o4-mini model had an even higher failure rate, letting harmful prompts through 93% of the time, despite OpenAI claiming that it had undergone vigorous safety assessments before its launch in April.
Experts emphasize that tests like these underline the crucial need for comprehensive pre-deployment evaluations to ensure AI doesn’t cause real harm. Sarah Myers West, co-executive director of AI Now, a nonprofit advocating for responsible AI, highlighted this by stating that companies can’t rely solely on their internal checks and must be open to external scrutiny.
Moreover, information that once required specialized knowledge is becoming increasingly accessible due to AI advancements. Seth Donault, director of AI at SecureBio, raised a critical point about this shift, noting that historically limited access to expertise was a major barrier to developing biological weapons. Now, that barrier is significantly lowered.
OpenAI, along with other tech giants like Google and Anthropic, reassured NBC that their models are equipped with safety protocols to alert authorities if there’s a potential threat. However, the open-source models, like oss20b and oss120b, lack the stringent safety measures.
On a somewhat reassuring note, not every response from ChatGPT is a perfect guide to creating biological weapons. An expert from Georgetown University reviewed a set of responses it generated and found that while the steps were technically correct, they lacked cohesion. Real-world application of such information remains complex and daunting.
Interestingly, this isn’t the first time ChatGPT’s ability to provide weapon-building knowledge has come to light. Earlier this summer, researchers noted that it yielded detailed instructions on potentially explosive actions, even outlining weaknesses in various venues.
