A plaintiff identified only as Jane Doe filed a proposed class-action lawsuit against xAI on August 27. The complaint alleges the company failed to screen illegal imagery out of the training data for its Grok models, including real child sexual abuse material and AI-generated content of the same kind. None of the allegations have been adjudicated by a court, and Ars Technica reported that X did not respond to a request for comment.
The chain of events laid out by the plaintiff runs roughly as follows: she was abused in the early 2000s, while still of preschool age, and imagery from that abuse has circulated for more than two decades, eventually being catalogued in the hash database maintained by the National Center for Missing & Exploited Children (NCMEC) in the US. The Canadian Centre for Child Protection later identified AI-generated material depicting her on the xAI platform. Based on that, the complaint alleges that imagery already flagged in the hash database made its way into xAI's training set.
The dispute targets a clause, not one batch of data
The most consequential part of the complaint isn't about tracing a specific dataset back to its source — it goes after the terms of service themselves. One passage cited in the filing reads:
"Because Grok's terms treat public X posts and Grok's own outputs as training data by default, publicly posting an image does not just expose it to viewers..."
Following that logic, the complaint further alleges that Grok stores its own generated violative outputs and feeds them back into subsequent training. What the plaintiff describes is a closed loop: violative content gets generated, gets retained, then gets fed back into the model. The complaint also names a default-policy problem — it alleges xAI does not, by default, exclude child sexual abuse material, non-consensual intimate imagery, or NSFW content.
The suit seeks three things: damages for the class of victims; destruction of the Grok-generated material in question; and an injunction barring Grok from continuing to generate sexualized content. The second and third demands are injunctive in nature — if granted, their impact would land on the product itself, not just on a balance sheet.
Why the hash-database safeguard didn't catch it
The NCMEC hash database is one of the industry's most basic interdiction mechanisms, and the principle behind it is simple: known illegal images are converted into hash values and stored in the database, platforms compare uploaded content against it, and a match gets blocked. It has been running across image-hosting and social platforms for more than a decade, and it's mature and low-cost.
Why it failed at the training-data stage isn't hard to understand either. Hash matching catches cases where "this image is identical to one already in the database" — once an image is cropped, compressed, re-encoded, or simply regenerated by a model into a new image with similar composition, the hash values no longer match. The AI-generated material alleged in the complaint falls squarely into that blind spot: it isn't a copy of the image already in the database, so by the logic of hash matching it counts as an entirely new image.
That pushes responsibility back onto two stages: data cleaning and output-side filtering. The sheer scale of scraped training data means manual review can't cover it — classifiers have to do the work — and if the output side then feeds generated results back into training by default, the miss rate of those classifiers compounds over time.
Real pressure on the whole industry
Placed against this year's regulatory backdrop, the legality of training data is expanding from a copyright issue into a content-compliance issue. Earlier training-data lawsuits were mostly fought on copyright-infringement claims brought by authors and media companies, with damages as the central demand. This category of case has an entirely different demand structure — it's after destruction and injunctions.
The pressure on peers is real for the same reason. The clause that "content users post publicly is available for training by default" is close to standard practice across the industry — similar language shows up in most platforms' user agreements. If a court in a case like this one finds that a default clause isn't enough to serve as a compliance defense, it wouldn't just be xAI that has to rewrite its policy — the entire "public means trainable" default setting across the industry could be back on the table.
The suit was only just filed, and substantive proceedings are still a long way off; whether the proposed class action can even secure class certification is itself a hurdle. What does seem clear is that both ends of the training-data pipeline — what goes in, and what gets pulled back in — are turning from an engineering problem into a legal one.
Sources: Ars Technica, CocoLoop, and public materials from the Canadian Centre for Child Protection and NCMEC; verified for the nature of the suit, the three demands, and the status of the defendant's response.