There’s yet another reason to be frustrated with technology.
A recent article in a journal highlighted that prominent AI chatbots, such as ChatGPT, struggle to differentiate between belief and fact, which raises red flags about the potential for spreading misinformation. The study from Stanford University observed that “most models lack a firm understanding of the factual nature of knowledge; that truth is an inherent requirement.”
The researchers noted that these bots tend to display inconsistent reasoning, likely due to superficial pattern recognition rather than a solid grasp on knowledge itself.
This inability to discern fact from fiction poses significant risks, especially as these technologies become commonplace in critical areas like medicine and law, where the capacity to differentiate between true and false is paramount. If these systems can’t identify accurate information, it could lead to erroneous medical diagnoses, skewed legal decisions, and a wider spread of misleading content.
To evaluate how well chatbots could discern truth, researchers analyzed 24 large-scale language models, including Claude, ChatGPT, DeepSeek, and Gemini. They posed about 13,000 questions designed to gauge the bots’ ability to differentiate between belief, knowledge, and factual information.
Overall, the findings indicated that machines were generally poor at distinguishing true beliefs from false ones, with older models displaying even weaker performance.
Interestingly, models introduced after May 2024, such as GPT-4o, had an accuracy of 91.1 to 91.5 percent for recognizing true and false statements, while older models ranged from 84.8 to 71.5 percent.
The conclusion suggests that bots struggle with the essence of knowledge, relying on inconsistent reasoning, which implies a shallow understanding rather than a deep comprehension of epistemology. It’s still somewhat surprising that large-scale language models have only recently shown they barely capture reality. Just recently, British innovator David Grunwald mentioned he inspired Mr. Grok to create a “poster for the last 10 British prime ministers,” but there were glaring inaccuracies in the final product, including misidentifying Rishi Sunak as Boris Johnson and incorrectly stating Theresa May served from 5747 to 70.
The study underscores the urgent need for improvements in AI before it can be considered trustworthy enough for critical fields like law and medicine, where accurate fact differentiation is crucial.
Pablo Haya Coll, a computational linguistics expert not involved in the study, suggested that training AI to respond more cautiously could be a way forward. She cautioned that these shortcomings could lead to significant errors in judgment, particularly in law, medicine, and journalism, where conflating belief with knowledge can have dire consequences.
As our reliance on AI for information grows, a report from Adobe Express this summer indicated that 77% of Americans using ChatGPT treat it like a search engine. Alarmingly, some users even trust it more than traditional search engines.
The concern is that this trend may make the public more vulnerable to subpar, misleading AI-generated content.
In a related incident from May, a California judge imposed a fine of $31,000 on two law firms after they used false information produced by AI in their legal documents without thorough verification.
