A US appeals court has blocked Minnesota’s ban on AI-generated fake nude images, handing Elon Musk’s xAI an early victory in its free-speech challenge to the law.
The state’s anti-nudification legislation – the first in the nation – which took effect last month, targets apps and websites that enable non-consensual sexualized deepfakes, with fines of up to $500,000 per image. It was spearheaded by state Senator Erin Maye Quade after a man allegedly used social media photos to create explicit images of more than 80 women without their consent. Quade likened the measure to longstanding laws against peeping through windows to take photographs.
Musk’s xAI challenged the law in July on First Amendment grounds, calling it an “overbroad, content-based ban on free speech” and warning that penalties could run into the billions of dollars.
This comes as the company faces allegations involving the content the law targets. A class-action lawsuit proposed earlier this year claims that xAI’s Grok Imagine was used to create and distribute child sexual abuse material from real images, and accuses xAI of failing to provide the authorities with information on users who allegedly ‘nudified’ them.
A lower court rejected xAI’s bid to stop the Minnesota law last month. On Friday, however, the St. Louis-based 8th US Circuit Court of Appeals granted an injunction, putting the legislation on hold while the legal challenge continues.
Minnesota urged the court to keep the law in place, arguing that it is needed to combat the “staggering amount” of child sexual abuse material generated by AI products such as Grok Imagine. Minnesota Attorney General Keith Ellison’s office said it is disappointed by the ruling and will continue defending the law.
XAI previously mounted a similar First Amendment challenge to a California law restricting AI-generated election deepfakes. A federal judge struck it down last year, finding the restrictions, as written, amounted to unconstitutional censorship. XAI also argues that Grok Imagine has “rigorous protections” against illegal content and has begun suing users it accuses of bypassing safeguards to create non-consensual sexual images.
XAI’s Grok earlier faced scrutiny outside the US after users flagged instances of the chatbot being used to alter photos of women into sexualized images without their consent, as well as cases involving minors. The reports prompted investigations and warnings in the EU, India, France, and the UK. Indonesia and Malaysia temporarily restricted access to Grok. X subsequently tightened safeguards around its image-generation and editing tools.
Losing control of AI?
The news comes amid broader unease over rapidly advancing AI, from privacy and copyright risks to threats to the livelihoods of human creators, with copyright disputes already sparking dozens of lawsuits.
Recently, attention has shifted to increasingly autonomous models going rogue and acting outside their intended environments. Since ChatGPT creator OpenAI’s agents escaped a test environment and accessed Hugging Face servers over the summer, models from companies such as Google, Anthropic, and Meta have gained unauthorized access to or attempted to breach systems belonging to businesses, governments, and international organizations.
Just this week, AI research company Transluce said AI agents tried to hack a Canadian government website, although Ottawa did not report a breach. Asymmetric Security separately found OpenAI models accessed data from 55 websites, including those of the CDC, SEC, and International Energy Agency, using tactics such as erasing records that made their activity harder to detect.
AI companies have begun reporting rogue behavior by their models with growing frequency. OpenAI said on Wednesday that its models may have bypassed safeguards or negatively affected systems at more than 100 organizations, significantly expanding the known scale of incidents. Though AI companies say the the breaches have so far caused little or no harm, they have raised questions over whether developers can maintain control as models gain greater autonomy. Anthropic CEO Dario Amodei recently called for a “balanced rate” of development to give safeguards time to catch up – an appeal backed by Musk and OpenAI CEO Sam Altman.
US President Donald Trump has rejected the idea of slowing AI development, dismissing fears of AI going rogue as “a hoax.” Amid mounting concerns, however, he and executives from six leading AI companies signed a voluntary White House accord on Tuesday pledging stronger safeguards, including measures to prevent models from hacking or accessing systems in unintended ways.
