Thousands of reports to NCMEC's CyberTipline last year were about suspected CSAM created using AI.
gettyThe mainstreaming of AI image generators has brought with it an influx of artificial child sexual abuse imagery, according to a new report from the National Center for Missing and Exploited Children.
The nonprofit—a go-between that funnels information about suspected CSAM and child sexual exploitation from tech and social media companies to law enforcement globally—received 36.2 million such reports to its “CyberTipline” in 2023, up from north of 32 million the year prior and more than double the pre-pandemic volume in 2019. Of those 36.2 million, NCMEC determined almost 5,000 to be the result of generative AI, though it says the actual count is likely considerably higher.
“It's fairly small volume when considering the overall total of the CyberTipline reports, but our concern is that we're going to continue to see that grow, especially as companies get better at detecting and reporting,” Fallon McNulty, director of the CyberTipline at NCMEC, told Forbes.
“What is really scary for us is, if we think about just the scale: It's so small and already has made such a huge impact,” she added.
Over the past year, there has been a pronounced uptick in reports of AI image generators being used to create illegal sexual abuse images. Deepfake nude photos of students are roiling middle and high schools across the United States. Some of the most popular new AI tools on the market have used illegal CSAM to train their models. And prosecutors recently charged a man in one of the first criminal cases involving AI-generated CSAM.
“The scale of this could explode.”
Recently, a small-but-growing group of generative AI shops have begun to cooperate with NCMEC to track and flag apparent CSAM, McNulty said. Leading the charge is OpenAI, creator of ChatGPT and text-to-image generator Dall-e, which started engaging with NCMEC last year and is now appearing in its report for the first time. Anthropic and Stability AI have also joined the effort, McNulty added, which is starting to paint a clearer picture of how most of this content is being made—whether by inputting text prompts, for example, or by using AI to manipulate photos of children or known CSAM. Although the biggest social media apps are being used to spread it (and roughly 70 percent of tips related to AI CSAM were submitted by mainstream platforms), ground zero is often open source models or off-platform, McNulty said. Some reports that NCMEC has received from social media companies include posts, comments, hashtags or chat logs detailing what AI model had been used.
With AI developing so rapidly and proliferating so quickly, McNulty said she fears “the scale of this could explode”—further inundating a disperse law enforcement system already struggling to keep up. That it is becoming increasingly difficult to distinguish fake AI-generated CSAM from the real thing is another challenge. And although a handful of big generative AI players have agreed to work with NCMEC, some of the smaller platforms often reported to the organization— like apps that can “nudify” a photo, for example—have not, McNulty noted.
The trend continues—in the first quarter of 2024, NCMEC has seen roughly 450 reports a month of CSAM stemming from AI—and is only expected to grow.
MORE FROM FORBES
