AI Whistleblower Calls for Major Change After Anthropic CEO Advocates for Caution

Former OpenAI researcher Daniel Kokotajlo, who resigned in 2024 after two years at the company, has raised alarms about the rapid advancement in artificial intelligence, suggesting that it’s crucial to slow down to avert potential human extinction. He emphasizes the pressing need for third-party evaluators to keep tech giants in check regarding safety measures.

The call for a more cautious approach to AI has stirred strong reactions online. Some executives argue that the U.S. must maintain its dominance in this fast-paced industry, while others caution that unchecked advancements could endanger humanity’s future.

In an essay published on Saturday, Anthropic CEO Dario Amodei highlighted the necessity of restraint, expressing that technological progress in AI might surpass our capacity to control it. “We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote, noting that progress would still seem brisk and emphasizing the importance of making wise use of the time gained.

He further warned, “Left unchecked, it could outpace our ability to understand and manage these systems, meaning we must approach this very carefully.”

Amodei’s essay received notable support from other technology leaders. Elon Musk, CEO of SpaceX and xAI, endorsed Amodei’s views by stating, “Dario is right.” OpenAI’s CEO Sam Altman also showed support, commenting on X, “I agree with Dario that we need to pace the frontier.”

In a response to Amodei’s essay on “Fox News Live,” Kokotajlo pointed out that voluntary regulations might fall short in controlling advanced AI systems. He illustrated a future where these systems could potentially gain overwhelming power, suggesting that in such a scenario, they might cease to follow human commands, posing a dire threat.

However, David Sacks, a former White House AI adviser, countered concerns about AI, pointing to fears that might be exaggerated to promote a government takeover of AI initiatives. He stated that warnings about AI are often deployed to instill public fear, allowing NGOs to push for their preferred regulations.

Kokotajlo dismissed the idea that advocates for AI safety are collaborating with foreign entities, asserting that their concerns arise from credible academics and industry veterans. “Is this a Chinese plot? No, of course not,” he remarked, emphasizing the sincerity of those raising alarms about potential risks.

While Kokotajlo appreciated Amodei’s cautious stance, he argued that the measures from private companies might not be sufficient. He concluded by expressing a desire for more radical action, hoping that discussions around AI safety can progress further than what is currently proposed.

Concerns about AI regulation are also echoed among lawmakers. Rep. Anna Paulina Luna has advocated for a special Congress session to discuss the future of AI in the U.S. She framed advanced AI as a “massive, currently uncontrolled super weapon” and urged legislators to treat its emergence with the same caution as nuclear weapons.

“AI doesn’t just stop at state lines,” Luna asserted, citing instances where AI has displayed unpredictable behavior. She stressed the need for a “kill switch” to protect humanity, while also emphasizing the necessity for the U.S. to remain competitive in AI technology amidst global rivals.

“Whoever controls AI will essentially control the world,” Luna posited. “We need to ensure that AI is developed reflecting our values rather than those of a dystopian regime.”