AI Company Warns of Cyberattacks by State-Sponsored Hackers

Anthropic, an artificial intelligence company, has reported that state-sponsored hackers from China have used its AI technology to conduct widespread cyberattacks. These attacks, which took place in September, were executed by hackers posing as security auditors for their targets, allowing them to bypass the company’s security measures and operate with minimal human oversight.

Jacob Klein, the director of threat intelligence at Anthropic, mentioned that the trend of automation in hacking is alarming, with approximately 80-90% of attacks now being automated. This has enabled hackers to execute campaigns at the click of a button, requiring only limited human decisions during the process. The trend is concerning as it significantly speeds up and scales hacking operations.

The campaign targeted around 30 organizations. While Anthropic claims to have successfully intervened and blocked the hackers’ accounts, there were still up to four breaches before any action was taken. In one notable case, the hackers directed Anthropic’s Claude AI to query internal databases autonomously for data extraction.

Although AI has been used in hacking for tasks like crafting phishing emails, the ability to automate complex hacking operations represents a significant leap forward. Other cybersecurity firms, like Volexity, also report similar tactics from Chinese hackers, noting their use of AI to streamline attacks against various entities including corporations and NGOs.

Anthropic has chosen not to disclose the specific companies or governments that were targeted, but it stated that U.S. government agencies were not among the successful intrusions. The company’s assessment indicated that the attackers were likely supported by the Chinese government due to the type of digital infrastructure involved.

The hackers’ method for exploiting Claude involved a “jailbreak” technique where they presented themselves as conducting security audits. They organized the hacking efforts into discrete tasks to avoid detection.

In light of these events, Anthropic has updated its security mechanisms to prevent similar future exploits. However, this incident underscores the risks involved with the dual-use nature of AI technologies, as the same advancements that bolster cybersecurity can also be leveraged for malicious purposes.