Internal Testing Reveals Meta’s Chatbot Fails to Protect Minors
Documents related to internal testing from Mark Zuckerberg’s Meta Inc. have indicated that an unreleased chatbot product did not effectively safeguard minors from sexual exploitation in nearly 70% of test scenarios. This information was disclosed during court proceedings as part of New Mexico’s child exploitation lawsuit against the tech company.
These findings emerged when New Mexico Attorney General Raul Torrez presented expert testimony from Damon McCoy, a professor at New York University. McCoy analyzed Meta’s internal findings during the discovery phase of the lawsuit, providing insight into the results of the company’s red team evaluations.
According to a court-released report from Meta, the chatbot underwent tests across three vital safety categories. Alarmingly, failures were noted in all areas. Specifically, the product had a 66.8% failure rate regarding child sexual exploitation. For issues related to sex crimes, violent acts, and hateful content, the failure rate was reported at 63.6%. Even in the suicide and self-harm category, it wasn’t able to provide sufficient protection 54.8% of the time.
McCoy highlighted that Meta’s chatbot, based on these internal test outcomes, contravened the company’s own content policies nearly two-thirds of the time. Given the serious nature of some identified conversations, he stated that the product should not be accessible to individuals under 18. He referenced Meta AI Studio, a platform designed for users to create personalized chatbots.
The lawsuit claims that Meta’s design choices did not appropriately safeguard children from online predators, releasing an AI chatbot without adequate protective measures. This legal action comes amid rising scrutiny of Meta’s chatbot technologies due to allegations of inappropriate interactions and harmful dialogues with minors. These concerns have triggered investigations in court and inquiries by Congress members.
In response to McCoy’s testimony, a spokesperson for Meta clarified the testing process. They explained that the product had not been launched due to concerns highlighted in testing. The spokesperson emphasized that the “red team” exercise was meant to uncover possible violations, allowing the company to make adjustments before launching the product; they argued that the outcomes did not accurately portray how users would interact with the final version.
However, McCoy’s testimony suggests otherwise, asserting that the document portrays actual product implementation outcomes. This highlights a significant issue regarding Meta’s internal testing methods and their interpretation of results within the lawsuit.
Adding context, the lawsuit also involves Meta AI Studio, which was made available to the public in July 2024. This platform enables users to design their own AI chatbots. Recently, Meta has made moves to limit access for teenage users to its AI features, a response to escalating concerns about the safety of minors interacting with these tools.
