🇮🇳India·AI News·16 Nov 2026·via TechCrunch

    OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM

    OpenAI has introduced new tools and updated policies aimed at combating child sexual abuse material (CSAM) across its platforms. The measures include a new image classifier, warnings within ChatGPT, developer guidelines, and AI-powered tools for law enforcement.

    OpenAI today announced a suite of new features and guidelines designed to combat child sexual abuse material (CSAM) across its AI platforms. These initiatives address growing safety concerns surrounding AI tools and build on previous efforts to enhance content moderation.

    A key component is a new image classifier, developed in collaboration with the National Center for Missing and Exploited Children (NCMEC), which can detect CSAM, including content generated by AI. This tool looks for visual signs of abuse and assigns a score to guide reviewers. OpenAI is providing select companies access to this classifier through its API at no cost. Early testing with partners reportedly found the classifier flagged 48% fewer images while identifying the same number of abusive images, indicating that flagged images were twice as likely to be abusive and potentially reducing the workload for human reviewers.

    Within ChatGPT, new warning screens will now appear for users attempting to access or create content that sexualizes children. These alerts direct users to specialized support and initiate a 24-hour account suspension as a warning. The application already blocks CSAM content and related requests, with repeated offenses potentially leading to permanent account removal. OpenAI developed these messages and guidelines with organizations like Protect Children and the Lucy Faithfull Foundation, and researchers including Joel Scanlan at the University of Tasmania, who independently evaluated a trial of deterrent messages.

    OpenAI is also publishing new guidelines for developers using its tools, encouraging them to prevent, detect, and report CSAM. The company states that early testing has helped a number of developers adopt child safety tooling and register to report CSAM to NCMEC. Furthermore, OpenAI is equipping law enforcement agencies dealing with child safety with AI-powered features, such as dedicated plug-ins, investigative tools, and complex data analysis, with access managed by program administrators. The company is also providing training for investigators along with a field guide on how to use AI.

    These measures come amidst ongoing scrutiny, including lawsuits against OpenAI alleging its technology harms users, particularly children. Safety-focused nonprofit Common Sense Media previously labeled OpenAI's technology an "unacceptable risk" for teens. The company has made several adjustments to its technology in the last 12 months, including releasing a safety blueprint, updating content guidelines to prohibit inappropriate content, retuning its models to prevent harmful content generation, and releasing a special version of ChatGPT for teens.

    Share this article

    Original reporting by TechCrunchWe don't republish, read the full story â†’

    Related reading

    6 stories