Researchers in the field of Artificial Intelligence are contemplating the potential creation of a “new species” of incredibly intelligent machines. However, there’s a shared concern that the US must not permit China to take the lead in this area.
In a recent debate, two seasoned researchers—one regarded as an AI “optimist” and the other as a “doomer”—came together to discuss issues like trust in China, ways to avert potential disasters, and the ineffective nature of “self-regulation.”
Oren Etzioni, a professor at the University of Washington and an AI entrepreneur, reflected on his views, admitting, “I was wrong. I still maintain some optimism, but it’s clear that the developments I thought were far off are happening much sooner than anticipated.”
Max Tegmark, founder of the Future of Life Institute, expressed his surprise that regulatory bodies haven’t intervened more decisively. He remarked, “I never imagined that the most powerful people in America would allow companies to approach such potentially uncontrollable advancements without taking action.”
One of the main alarms raised was about “recursive self-improvement,” where AI reaches a point where it can innovate and enhance itself, potentially outpacing humans in AI research.
Yet, both experts agreed that a straightforward solution to prevent AI from exceeding safe boundaries is to keep its functionalities separate, assigning different machines to distinct tasks.
Etzioni has consistently advocated for AI systems with narrowly defined expertise rather than broad-ranging superintelligent agents. “We need to design systems with caution to keep them contained,” he stressed.
Tegmark concurred, stating, “Machines should be granted the minimal access and power necessary for their tasks.” He illustrated this by likening it to not giving every custodian in the Pentagon access to nuclear codes.
Regarding regulatory efforts, the Trump administration has insisted that top AI corporations should independently manage their creations. Nonetheless, Trump did announce plans for a new AI adviser and the formation of an “AI Force” earlier this month.
Some AI companies, like Anthropic and OpenAI, have recently called for stronger regulations, which smaller enterprises fear could hinder innovation in their sector.
Dario Amodei, CEO of Anthropic, proposed that the leading companies should have a say in how they are regulated, a suggestion both Tegmark and Etzioni found absurd. “Self-regulation is a joke,” Etzioni remarked, likening the situation to the failures of social media companies in the past. He expressed concern that this moment parallels the “big tobacco” crisis for high technology.
Both Tegmark and Etzioni, who co-founded a venture acquired by Anthropic this year, advocate for a regulatory body, similar to the FDA, to ensure AI technologies are helpful rather than harmful. “We should hold AI companies to standards like other industries before they get rich,” Tegmark proposed, believing this would incentivize safety in products.
Interestingly, after a long period of pessimism, Tegmark feels “much more optimistic” now, noting that anxiety about AI is becoming mainstream.
Etzioni, however, expressed his worry that a hasty regulatory response in the US might give China an advantage. “I don’t trust the Chinese,” he stated, cautioning that they would benefit from the US incapacitating itself.
Tegmark countered this sentiment, arguing that there might be a universal acknowledgment that superintelligent AI poses risks that transcend borders. “We don’t need to rely on China’s government to prevent their companies from taking reckless actions,” he emphasized.
Ultimately, the discussion highlighted a striking reversal in outlook: the typically optimistic figure found himself more cautious, while the usual pessimist embraced a more hopeful perspective. This shift underscores the frantic pace at which AI development is proceeding, catching even the experts off guard.
