OpenAI Launches New AI Tool to Detect AI-Generated CSAM, Adds 24-Hour ChatGPT Suspensions
OpenAI is stepping up its fight against online child exploitation with a new set of safety measures. The company has launched an image detection tool, stronger ChatGPT interventions, and additional support for developers and law enforcement. The announcement also puts AI-generated abusive material under greater scrutiny.
OpenAI makes CSAM detection tools available for free
The biggest announcement is a new image classifier designed to identify potential child sexual abuse material (CSAM), including AI-generated images. Unlike traditional detection systems, the classifier looks for visual signs of abuse and can flag previously unseen material.
OpenAI says vetted organizations can access the tool through its API at no cost. The company developed it with assistance from the National Center for Missing and Exploited Children (NCMEC).
Early evaluations show promising results. OpenAI estimates the classifier could reduce image review workloads by 48% while identifying roughly the same number of abusive images. Images flagged by the new system were also twice as likely to contain abuse.
Partner testing produced additional results. Fal confirmed 72% of high-confidence flags as novel CSAM, while SafetyKit detected 90% of tested drawings and animations sexualizing children.
ChatGPT gets stronger warnings and account restrictions
OpenAI is also introducing new interventions inside ChatGPT. When users attempt to find abusive material, the platform will display warnings explaining the harm and consequences.
The flagged conversation will be removed, and access to ChatGPT will be paused for 24 hours. Specialist support will remain available to help users stop harmful behavior. Repeated violations could result in longer restrictions, account closure, or reports to authorities.
Developers and investigators receive additional support
OpenAI has published practical CSAM safety guidance for developers. It covers detection, reporting, moderation, and reviewing suspected abuse. The company is also supporting specialist law enforcement teams with AI tools for reviewing lengthy chat logs, identifying connections, and building investigation timelines.
That being said, OpenAI says these efforts require broader industry cooperation. Its partnerships include NCMEC, Thorn, Tech Coalition, and ROOST. The company is also participating in the Lantern program to share signals about online child exploitation.
Read our disclosure page to find out how can you help Windows Report sustain the editorial team. Read more
User forum
0 messages