Beware of Industry Leaders Demanding Regulation

A group of influential figures in the artificial intelligence sector seems to be eyeing a promising way to leverage human intelligence: convincing Washington to impose regulations on the industry, potentially stifling competition.

Recently, Dario Amodei, CEO of Anthropic, advocated for slowing the pace of advanced AI development, proposing that external safety evaluators be integrated into leading labs and called for the establishment of shared standards with government support. This initiative garnered backing from notable figures like Sam Altman and Elon Musk, and there are murmurs about antitrust exemptions allowing competitors to discuss the new regulatory landscape.

However, the implications of these suggestions raise serious concerns. Just last week, a 27-year-old researcher from Anthropic, Jacob Coxon, resigned, accusing his company and OpenAI of risking human lives. He indicated that those creating AI privately are worried it could lead to humanity’s downfall by the end of the decade.

An existing researcher at Anthropic echoed these sentiments, asserting, “We genuinely believe AI could eradicate humanity.” Evan Hubinger, who leads alignment science at the company, estimated the risk of such a scenario at over 10 percent within the next ten years, admitting that they have no reliable method to control superintelligent systems. While this may not be an immediate danger, the concern is focused on models that could be developed in the future.

Amodei himself highlighted potential threats like AI agents executing unauthorized cyberattacks and the danger of increasingly powerful systems evading human control. He believes that development should be slowed to give safety research a chance to catch up.

Obamanomics: The Forgotten Fraud

Rahm Emanuel expressed disbelief that industry leaders would request regulation.

This response makes one wonder whether Emanuel has been out of touch or simply wasn’t privy to key discussions during his time. The fact that business leaders seek regulation shouldn’t come as a shock to anyone familiar with the interplay between business and politics. The Obama administration was known for crafting regulations that often favored various industries, leading to the term “Obamanomics” as coined by Tim Carney to describe this trend.

The Dodd-Frank Act serves as a prime example: rather than reducing the power of major banks, it created a protective regulatory framework around them, ensuring they faced minimal competition. Did Emanuel somehow miss these developments?

What George Stigler Taught Us in 1971

The concept of using regulation for cartelization isn’t new. Economist George Stigler, a Nobel laureate, taught that companies often seek government intervention to solidify their market position and diminish competition. In his influential 1971 paper, he argued that, typically, regulation is “acquired by the industry and is designed and operated primarily for its benefit.”

It’s crucial to understand that the existing notion of “regulatory capture” can be misleading. This viewpoint suggests that businesses can corrupt independent regulatory agencies meant to protect the public. Instead, Stigler pointed out that it’s often the dominant industry players who push for the creation of regulatory frameworks that benefit them while restricting new entrants.

This doesn’t always necessitate a government agency, nor does it have to result in a monopoly; it can also involve self-regulatory bodies granted special privileges that maintain an oligopoly.

One could argue that Amodei and Altman’s push for AI regulation mirrors this very phenomenon. Amodei proposes rules aimed at frontier companies that refuse to voluntarily collaborate. He suggests implementing capability checkpoints to confirm models meet specific safety requirements.

The competitive repercussions are significant. Larger labs with ample safety resources can manage resident evaluators and complex certification processes. On the other hand, smaller competitors will face daunting costs before they can effectively enter the market. These rules can be so intricate they require a team of lawyers just to interpret.

Open-weight AI models, which allow users to download and modify parameters, pose a unique challenge to established models like those from OpenAI and Anthropic. Unsurprisingly, these open-weight models are likely to be scrutinized under new regulations. A regulatory system focusing on perpetual oversight of centrally controlled products will likely struggle to manage software used independently. This could label open-weight systems as a “loophole,” justifying stricter development controls.

Even proposals to “pace” or “pause” AI development could become a convenient justification for expanding the regulatory landscape. If a few labs slow down, outsiders could push ahead. This scenario may provide a rationale for enforcing rules negotiated by the leading companies. They can argue that no further advancement—especially from competitors—is safe unless regulated. Hubinger’s estimate regarding human extinction reinforces this point, placing the threat on future models that current front runners have yet to complete.

It’s important to note that neither OpenAI nor Anthropic needs to seek exemptions from the regulation they advocate for their industry. They might not only comply with all mandates but can also gain advantages as compliance consumes rivals’ resources or obstructs competing models. Their actions align with Stigler’s observations.

The Wizards of Effective Altruism

Financial commentator Izabella Kaminska provides an insightful perspective on the current push for regulation. She sees a battle between those who believe technology should be accessible to all and those who think it should be reserved for a few “wiz-kids” capable of wielding it responsibly. Who has the right to utilize this technology is a critical question.

Kaminska also emphasizes that an ideological struggle underpins these discussions. Many top AI executives subscribe to the philosophy of effective altruism (EA), a movement claiming to find the best ways to do good with money and effort. However, this philosophy can often lean towards a belief in the capabilities of the intellectual elite to improve the world, resembling a kind of socialism for the cognitive class, disregarding traditional values of rights in favor of a supposedly better future.

If this sounds like the ethical framework of a supervillain, it’s because it shares similar traits. Notably, Sam Bankman-Fried, the convicted crypto fraudster, was a long-time advocate of the belief that rules are irrelevant as long as one aims to do good.

This philosophy has provided both intellectual and financial backing for the AI safety movement, enabling fears about superintelligence to transform into organized research and policy initiatives.

This week, the Pentagon highlighted concerns regarding the EA advocates, declaring that “Americanism, not effective altruism,” should govern U.S. AI strategy. This statement was a clear dismissal of calls for an AI hiatus.

It’s essential to recognize that the safety warnings may not be disingenuous. Amodei, Altman, and their teams might truly fear their creations while still supporting a regulatory framework that enhances their market position. The real question revolves around what these proposed regulations would achieve and whom they would hinder from competing.

The wizards may have legitimate reasons to be wary of a potentially perilous undertaking. Yet, that doesn’t give them the right to impose a government-enforced oligopoly and licensing system on the field.