OpenAI’s child safety tools aim to combat abuse created by AI.

OpenAI's child safety tools aim to combat abuse created by AI.

A teenager shares an ordinary photo online. Someone then uses AI to make a sexually explicit fake and circulates it among classmates. The image may eventually be removed, but the harm to the child can linger. It’s a troubling reminder that even a simple photo can be misused.

OpenAI says it is adding new measures to address child sexual abuse material, including images created with AI. The company is introducing a tool to help identify suspected abuse and changing how ChatGPT responds to attempts to find such material.

Many existing detection systems rely on digital fingerprints, or hashes, to recognize files that have already been identified as abusive. Newly created material may not have a matching fingerprint. OpenAI’s new classifier instead looks for visual signs of potential child sexual abuse material, which could help flag images that have not been seen before.

The tool assigns scores to images so human reviewers can prioritize cases. OpenAI says it developed and evaluated the classifier with help from the National Center for Missing & Exploited Children. It is making the technology available at no cost through its API to vetted organizations, potentially giving smaller child-safety teams access to tools that may otherwise be out of reach.

In early testing, OpenAI compared the classifier with its existing protections using images uploaded to its systems. The company says the new tool could reduce the number of images sent for review by 48% while detecting roughly the same amount of abusive material. Images it flagged were about twice as likely to contain abuse.

Other organizations have reported their own results. AI platform Fal said reviewers confirmed that 72% of its high-confidence flags involved previously unidentified child sexual abuse material. SafetyKit reported that the classifier detected 90% of cases involving drawings or animations that sexualize children in its evaluation. These figures come from OpenAI and participating organizations; independent testing will be important in assessing how well the system works across different platforms.

OpenAI is also changing ChatGPT’s response to attempts to find child sexual abuse material. New warnings will explain the harm caused by sexualizing children, including through AI-generated images. The company says it will remove the flagged conversation and suspend access to ChatGPT for 24 hours, while directing the user to specialist support intended to help prevent harmful behavior.

Repeated violations could lead to longer suspensions, permanent account closure or reports to authorities. OpenAI says its existing policies already ban accounts and require reports to NCMEC when people try to generate or upload child sexual abuse material. The new warnings were developed with input from child-protection organizations and researchers. Whether they help prevent future abuse will need to be assessed over time.

The company also says it has joined the Tech Coalition’s Lantern program, which lets participating organizations share information about suspected online exploitation. OpenAI is developing AI tools and training for child-safety law enforcement teams. Its first investigative tool is intended to help authorized investigators review lengthy chat logs and records obtained through legal processes, identify possible connections and organize timelines. Findings will be linked to their source material, and access will depend on agency authorization and program requirements.

OpenAI is also publishing guidance for developers on identifying suspected abuse and setting up procedures to review potentially harmful content.

Families can’t control every way someone might misuse a child’s photo, but open conversations can make a difference. Children should know they can tell a parent or trusted adult what happened without automatically losing access to their devices. That may make it easier for them to speak up before harmful material spreads further.

The new classifier operates behind the scenes, so its impact will depend on how widely organizations adopt it and how quickly their safety teams respond. The scale of the problem is significant: NCMEC says its CyberTipline received more than 400,000 reports involving generative AI in 2025. Those reports covered several types of AI-related child exploitation; not all involved the creation or sharing of AI-generated abusive images.

Making the classifier available to vetted organizations for free seems like a useful step, particularly if it helps investigators spend less time on false alarms and more time identifying children who need help. The 24-hour suspension also gives ChatGPT’s warnings a consequence beyond simply rejecting a request. Still, the technology will only be as effective as the systems and people put in place to use it. AI companies will need to keep adapting as their tools are misused.

Facebook
Twitter
LinkedIn
Reddit
Telegram
WhatsApp

Related News