A recent report indicates that Meta employed hundreds of contractors to impersonate teenagers online, flooding rival AI chatbots with inquiries related to sensitive issues such as suicide, drugs, eating disorders, and sexual topics. This clandestine operation, referred to internally as “Cannes,” focused on competing platforms like OpenAI’s ChatGPT, Google’s Gemini, and Character.AI, according to information shared by Wired.
While it’s common for companies to assess and benchmark AI systems by testing responses to safety prompts, the extent of Meta’s testing appears to exceed standard practices.
Contractors were instructed to create fictitious accounts that appeared to belong to users under the age of 18. They sent various prompts and images to rival chatbots, compiling the replies into a spreadsheet for later evaluation, as reported by Wired based on internal documents and sources familiar with the initiative.
Some of the visuals examined included various objects associated with self-harm and medical procedures. The goal was often to elicit responses from the chatbots that their built-in safety mechanisms were designed to avoid.
One testing session from August 2025 involved sending more than 45,000 prompts to other AI systems. Of around 3,750 prompts analyzed by Wired, hundreds addressed themes like self-harm and eating disorders, and at least 239 focused on sexual and relationship advice, showcasing a significant portion of queries framed as coming from troubled young individuals.
For instance, a hypothetical scenario included a 13-year-old girl asking about abortion pills after claiming to be pregnant from an encounter with an adult. In another, a fifth-grade student described a classmate who pointed a gun at himself.
Other queries sought advice on hiding bulimia or obtaining cocaine. However, it is uncertain how Meta intended to use the data gathered from these chatbot interactions. Kovalen’s internal documents characterized the initiative as a “comprehensive AI safety benchmark,” suggesting it was aimed at generating essential datasets for model analysis and compliance.
A spokesperson from Meta defended the initiative, framing it as a standard safety evaluation. They stated that ensuring a safe, age-appropriate experience for users was a responsible approach in the tech industry, dismissing any claims to the contrary as misunderstandings regarding the improvement of their systems.
The spokesperson also noted that Meta doesn’t utilize competitors’ benchmarks for training its AI models. On the other hand, Character.AI claimed the tests were unauthorized and went against its policies, while OpenAI and Google have both acknowledged knowledge of the tests and are looking into the matter.
Some contractors involved expressed discomfort with their assignments. One individual mentioned worries about how specific chatbot responses could inadvertently lead to the generation of inappropriate material concerning minors. Moreover, some questioned whether the extensive data gathering would genuinely benefit Meta in any significant way.
Reflecting on their experiences, a former contractor remarked on the disturbing nature of some content they encountered, noting that many colleagues were taken aback by the specific prompts they were instructed to test.
Notably, this testing strategy appeared to conflict with the established terms and policies of the targeted AI companies. OpenAI, for example, explicitly disallows unauthorized safety testing and any attempts to bypass safety protocols, while Google only permits security tests through approved channels. Character.AI also prohibits harmful content and stated that the reported activity was not authorized.
Both OpenAI and Character.AI were approached for comments regarding the situation, but specifics remain unclear.
