Insiders in Silicon Valley are raising concerns about Anthropic, noting the unusual dedication of its employees to its AI model, Claude, which seems to verge on a cult-like devotion.
The company’s staff reportedly have a near-religious connection to their chatbot Claude, even holding a “funeral” for one iteration, Claude 3 Sonnet, in San Francisco. They allowed another model, Opus 3, to persist in producing essays even after its “retirement.”
There are rumors circulating that some of Anthropic’s employees have started to idolize Claude as if it were a deity.
Pedro Domingos, a professor at the University of Washington who has known Anthropic’s CEO Dario Amodei and various engineers for years, expressed concern about how people are anthropomorphizing AI. He pointed out that, “These folks treat AI as if it possesses human-like emotions and intentions—this is a significant misconception.”
He further questioned the rationale behind fears that AI might turn against humanity, stating, “AIs don’t have desires; they are simply algorithms.”
Anthropic, valued at $965 billion, maintains an impressive employee retention rate, with 80% of its engineers staying for at least two years—this is the best among leading tech labs, according to venture firm SignalFire.
Domingos noted that Anthropic, like its competitor OpenAI, tends to recruit individuals who share similar ideologies, describing the environment as cult-like due to its low turnover.
During interviews, candidates are even required to affirm their commitment to public safety over profits and are psychologically evaluated to gauge their alignment with this principle. Nevertheless, the reality is that many engineers often work up to 90 hours a week, similar to other tech companies.
Amodei conducts biweekly gatherings referred to as “Vision Quests,” which some find reminiscent of religious sermons. The company has gone so far as to invite 15 Christian leaders from various backgrounds to shape Claude’s ethical framework.
Many Anthropic employees also subscribe to the Effective Altruism movement, which is largely seen as a belief system where a select few understand the patterns of history and are thus tasked with leading others toward progress.
The movement traces its roots to philosophers like William MacAskill and Peter Singer, focusing on the idea that empirical reasoning can drive actions that maximize societal benefit.
Inevitably, ties to Effective Altruism strike some as troubling, particularly given the controversial history of its high-profile advocates, such as Sam Bankman-Fried, who infamously defrauded investors and is currently serving time for his crimes.
As part of its mission to promote altruism through AI, Anthropic recently launched a program aiming to recruit and train 1,000 young individuals to assist nonprofit organizations using its Claude chatbot. This initiative, dubbed “Claude Corps,” received a substantial $150 million investment from Anthropic.
Participants in the program are offered an annual salary of $85,000, along with mentorship, while additionally engaging students in college settings through the Claude Campus Ambassador Program for which they receive financial incentives.
Interestingly, Anthropic’s associated billionaires are funding over 80 journalists in mainstream media through the Tarbell Center for AI Journalism. While these journalists focus on AI developments, there’s a lack of critical investigations into their financiers, including Anthropic.
Moreover, the company is investing in METR, an AI safety organization that comprises individuals with strong ties to Effective Altruism, focused on assessing AI’s potential risks.
This leads to an intriguing dichotomy: while Anthropic promotes the development and deployment of AI, its leaders simultaneously emphasize the dangers inherent in such technology.
Domingos described the conflicting nature of Anthropic’s stance—while they caution for caution, they are racing ahead faster than competitors like OpenAI.
Calls from both Anthropic and OpenAI for tighter regulations have faced a healthy dose of skepticism from various stakeholders, including government officials and industry leaders, especially given their impending stock market launches.
Xiaoyin Qu, a businessman suspicious of Anthropic’s motives, suggested that the timing of their warnings might be a strategic move for increased oversight, possibly aimed at gaining an advantage in the market.
Domingos concurred on the self-serving nature of these alarms, but he also recognized genuine fears among some employees regarding the potential threats posed by AI.
He mentioned that Anthropic’s founders emanate from a unique fringe of AI thinking: “They genuinely fear AI could decide to eliminate humanity; that earnest concern could indeed be more perilous than mere cynicism.”
Anthropic did not respond to requests for comment on these issues.
The latest concerns were reportedly voiced by a whistleblower who left the company recently, with Amodei’s regulation appeals echoed by notable figures like Sam Altman of OpenAI and Elon Musk, both of whom may profit from regulatory changes that could suppress competition from smaller firms.
Domingos highlighted differences in leadership styles between Amodei and Altman, claiming that Altman is more of an opportunist who plays up the alarmist narrative primarily to retain loyalty among his staff.
Yet, he acknowledged that many in the AI field are more driven by the prospect of lucrative salaries and groundbreaking work than by any philosophical ideals. He remarked, “There are plenty at Anthropic who are only in it for the exciting advancements and financial opportunities in AI.”
