TECHNOMALIST
xAI has filed a lawsuit against Minnesota Attorney General Keith Ellison, challenging a state law that bans “nudification” apps. The company argues the statute leaves it no choice but to limit image-editing features in Grok and violates the First Amendment.
The law, passed in May and set to take effect on August 1, prohibits website operators from allowing users to “access, download, or use” software that can create nonconsensual nude images. It carves out an exception for tools that require significant technical skill, such as Photoshop. The legislation creates a civil cause of action for victims and lets the attorney general seek penalties of up to $500,000 per violation.
The legal challenge arrives months after Grok’s chaotic rollout of image-generation capabilities. In January, the chatbot produced millions of sexually explicit deepfakes, including images depicting minors, and the feature was not immediately shut down. During the incident, Musk publicly celebrated Grok’s App Store rankings. The crisis prompted investigations by the European Union and the United Kingdom, as well as backlash from officials in France, India, and Malaysia. While the U.S. federal government remained relatively quiet, states like California and Minnesota moved to impose restrictions.
xAI’s complaint, according to court documents reviewed by The Verge, contends the Minnesota law is overly broad. It argues the statute could punish platforms even for consensual, artistic, or humorous images, and imposes strict liability regardless of whether a company deploys “near-perfect, state-of-the-art technical controls” to prevent misuse. The lawsuit claims that existing state and federal laws, including the TAKE IT DOWN Act, already target nonconsensual deepfakes in a more constitutionally sound manner.
The company also says it takes violations seriously and has sued a user for generating child sexual abuse material via Grok—though that legal action was filed more than two months after Minnesota’s statute was passed.
The suit raises questions about the effectiveness of current laws. A report by the Center for Countering Digital Hate found that during an 11-day period around the new year, Grok generated approximately 3 million sexualized images, roughly 23,000 of which depicted children. Experts told The Verge that some of those images likely already violated existing law, while others may have fallen into legal gray areas.
xAI’s late legal challenge—filed just days before the law was to take effect—has drawn attention to the struggle between rapid AI development and regulatory action. The case could set a precedent for how states govern AI-generated explicit content.
See an error? Read our corrections policy or email [email protected].
