OpenAI Report Reveals Chinese Use of ChatGPT in Information Warfare
OpenAI, the organization behind the ChatGPT AI system, shared a report on Wednesday detailing how a Chinese official reportedly utilized the AI to review and edit documents, which inadvertently exposed various global networks for potential blackmail against dissidents.
The report did not name the Chinese official involved, but it noted that while the account appeared to be used by an “individual,” it’s possible multiple users accessed it. It highlighted that this individual’s actions indicated a well-organized strategy aimed at covert information operations, termed as “cyber special operations,” against both domestic and international adversaries.
Interestingly, the report mentioned an attempt to leverage the AI model for a covert operation targeting Japan’s Prime Minister. However, OpenAI’s system rejected that request.
Japan’s Prime Minister, Sanae Takaichi, who has connections to the Chinese Communist Party, warned that a potential Chinese invasion of Taiwan might create an “existential threat” to Japan. This language holds weight, potentially allowing military intervention under Japan’s current pacifist constitution. So far, the Japanese Prime Minister has been careful with promises regarding intervention to safeguard Taiwan.
The report indicated that before he even took office, the Chinese user sought assistance from ChatGPT for an information warfare campaign against another figure due to past criticisms concerning human rights in Inner Mongolia. There was also a goal to spark outrage over U.S. tariffs through AI assistance.
OpenAI noted that Chinese threat actors were previously utilizing ChatGPT to generate regular status reports, which suggested an ongoing information warfare initiative against this individual and illustrated a comprehensive effort to silence dissenting voices both domestically and globally.
This assumed wide-scale operation entails significant resources, involving hundreds of personnel and thousands of fake online profiles on various platforms, and it appears to leverage primarily domestic AI technologies, particularly those developed in China.
In essence, the official’s choice to use ChatGPT to ease the editing workload for reports inadvertently laid bare some striking elements of China’s cross-border suppression plans.
For those familiar with oppression under the Chinese Communist Party, this might not come as a shock. Still, it’s somewhat awkward to see officials relying on AI to handle their administrative tasks.
OpenAI’s findings indicated that the data inputted into ChatGPT included details about various tactics for undermining dissent, including abusive social media reporting and creating fake documents to intimidate critics. Researchers noted online behaviors aligned with some of the tactics employed by this user.
In addition to the figure targeting the premiership, the campaign also focused on a Chinese social media user known as “Teacher Li Is Not Your Teacher,” or Li Ying, a recognized dissident.
If the information inputted to ChatGPT by the officials is correct, China’s information warfare strategy reportedly utilized thousands of accounts to produce millions of posts on domestic networks, alongside tens of thousands on international platforms—many of which were fabricated or controlled by the operation.
OpenAI uncovered some evidence suggesting that these campaigns had some success, as there was a noted decline in follower counts and online activity among targeted dissidents, with some accounts being permanently removed. However, renowned targets like the Japanese Prime Minister and Teacher Li did not appear significantly impacted.
“This illustrates what modern cross-border repression looks like in China. It’s not merely digital or just trolling. It’s industrialized and aims to confront critics of the Chinese Communist Party on all fronts at once,” stated Ben Nimmo, a Principal Scientist at OpenAI.
