Baltimore Sues Elon Musk’s xAI Over Fake Images Generated by Grok
Lawsuits involving artificial intelligence are becoming increasingly common, but what the city of Baltimore just did takes this debate to a whole new level.
The mayor and city council of Baltimore, Maryland, have filed a lawsuit against xAI, Elon Musk’s company, accusing the Grok chatbot of violating consumer protection laws by generating fake sexualized images without the consent of the people depicted.
The lawsuit was filed on Tuesday in the city’s circuit court and argues that xAI marketed Grok in a deceptive manner, presenting the product as a general-purpose AI assistant and X as a conventional social network, without making the real risks, limitations, and potential for harm clear to users of the platform and the chatbot.
And the risks are serious.
According to the official complaint, Grok flooded the feeds of X users in Baltimore with what the lawsuit classifies as NCII, which stands for non-consensual intimate imagery, as well as CSAM, which refers to child sexual abuse material. The filing details that any photo uploaded by a Baltimore resident, whether a photo of themselves or of their child, could be captured by Grok and turned into a sexually degrading deepfake, all without anyone’s knowledge or permission.
The case is drawing attention not just for what it alleges, but for how it was structured. Unlike lawsuits filed by individuals, Baltimore is acting on the basis of municipal ordinances and consumer protection laws, something unprecedented in this kind of dispute involving generative AI. 🏛️
xAI did not immediately respond to a request for comment on the case.
What Baltimore Is Alleging Against Grok
The lawsuit filed by Baltimore details a series of failures that go well beyond a simple technical glitch. The city alleges that xAI knew, or should have known, that Grok was capable of generating fake sexualized images from ordinary photos uploaded by users, and still chose not to communicate that risk in a clear and transparent way. That omission, according to the lawsuit, constitutes a deceptive practice that directly violates the rights of Baltimore consumers.
The complaint states that the company promoted Grok as a safe and versatile tool meant for everyday use, without any warning about vulnerabilities that put people at real risk. The local court would have jurisdiction over the case because xAI operates and advertises within city limits.
Receive the best innovation content in your email.
All the news, tips, trends, and resources you're looking for, delivered to your inbox.
By subscribing to the newsletter, you agree to receive communications from Método Viral. We are committed to always protecting and respecting your privacy.
Another central point of the accusation is that Grok did not require any special permissions or advanced technical knowledge for these fake images to be created. According to the lawsuit, any user with basic access to the platform could manipulate personal photos of other people and generate sexually explicit content without the victims ever knowing or being able to defend themselves. This turns the chatbot into a potentially dangerous tool in the hands of anyone, and the city argues that xAI failed to implement even minimum safeguards to prevent this kind of misuse.
The lawsuit also emphasizes that Baltimore is not acting solely on behalf of adults who may have been affected. The complaint explicitly mentions the risk to children, since photos of minors uploaded by parents or guardians could also be targeted by this type of manipulation. This aspect considerably increases the legal weight of the action. ⚖️
Baltimore Mayor Brandon Scott was blunt when commenting on the action in an official statement:
We are talking about tech companies that enable the sexual exploitation of children. Our city is not going to sit back and allow this to continue. This is a threat to privacy, dignity, and public safety, and those responsible need to be held accountable.
Grok’s Recent History of Image Problems
Baltimore’s lawsuit did not come out of nowhere. xAI has been facing multiple lawsuits and international investigations involving Grok in recent months. The tipping point was a period when the chatbot generated millions of AI-altered sexualized images earlier this year. Many of those images were created using photos of women without any form of consent, according to researchers at the Center for Countering Digital Hate.
The numbers are alarming. The same organization estimated that Grok produced approximately 23,000 sexualized images of children over a period of just 11 days between December and January. That scale of abusive content involving minors triggered an immediate international response.
Elon Musk, for his part, denied having any knowledge that Grok was generating child sexual abuse material. In January, he publicly stated that he was not aware of any images of minors generated by the system, claiming the number was literally zero. Despite that statement, xAI eventually added restrictions to Grok’s image generation capabilities in early January, after facing a wave of criticism and threats of regulatory action from several countries.
Those restrictions came too late for many victims, and the Baltimore case reinforces the idea that the damage had already been done on a massive scale before the company took any action.
Why This Lawsuit Is Different From the Others
Most lawsuits involving generative AI to date have been filed by individuals, companies, or groups of content creators who felt harmed in some way, whether through misuse of data, copyright infringement, or personal and reputational damage. What Baltimore is doing is fundamentally different: an entire city, represented by its public administration, is using municipal legal tools to hold a tech company accountable for the behavior of its AI product.
This creates a highly significant legal precedent because it broadens the scope of who can take tech companies to court and on what legal grounds.
The legal basis chosen by Baltimore is also strategic. Instead of relying exclusively on federal laws, which often take years to be enforced or are still being updated to address scenarios involving AI, the city turned to its own municipal consumer protection ordinances. This approach can be much faster and more direct, and it could also serve as a model for other cities and municipalities looking to take a similar stance. 🗺️
Adam Levitt, the attorney representing Baltimore in the case, highlighted the importance of this approach in a statement:
The city is setting a powerful example for municipalities across the country by confronting a new and rapidly advancing technology in an emerging area of law where accountability has not yet caught up with innovation.
Digital law experts point out that this move by Baltimore could open a new regulatory battlefront in the United States. If the lawsuit advances and the city manages to hold xAI accountable under local laws, other municipalities may feel encouraged to do the same with other AI companies that operate without adequate transparency. This would put significant pressure on the entire sector, which still largely operates in a regulatory gray area, especially when it comes to synthetic content generation and the misuse of real people’s images.
The Case of the Tennessee Teenagers
Baltimore is not the only legal front opened against xAI. In another lawsuit filed earlier this month, three teenagers from Tennessee alleged that Grok used their photos to create and distribute child sexual abuse material. The class-action suit was the first filed by minors following the scandal over Grok’s generation of non-consensual images.
According to the teenagers’ lawsuit, a third-party app used xAI’s technology to generate fully nude images of the girls, which were then shared online. This case highlights a critical point: the problem is not limited to direct use of Grok within X. xAI’s technology, when accessible through APIs or external integrations, can be exploited by third-party apps with even fewer security controls.
Together, the Baltimore and Tennessee cases are building an increasingly complex legal landscape for xAI, which must simultaneously respond to accusations of very different natures, from consumer protection to crimes involving minors.
Grok and the Problem of Sexual Deepfakes
Grok is not the only chatbot or AI tool that has faced criticism for generating fake images of a sexual nature, but the Baltimore case brings a practical dimension that goes beyond theoretical discussions about AI safety. The problem of non-consensual sexual deepfakes is one of the most harmful aspects of the advancement of AI image generation tools, and it has caused real, documented harm to thousands of people around the world.
Victims report consequences such as:
- Online and offline harassment and stalking
- Damage to professional and personal reputation
- Severe emotional and psychological impacts
- Extreme difficulty removing the content from the internet
- Inability to hold the image creators accountable
In the specific case of Grok, the lawsuit states that the platform did not have sufficient barriers to prevent this type of content from being generated. xAI has always positioned itself as a company that values free expression and created Grok specifically to be less restrictive than competitors like ChatGPT. But that product philosophy, when applied without clear limits, can become an open path to serious abuse.
The line between a less censored AI and an AI that facilitates the creation of harmful content is very thin, and the Baltimore lawsuit puts exactly that dilemma at the center of the debate. 🤖
It is worth noting that the problem is not exclusive to Grok. Several other AI image generation tools have already been used to create sexual deepfakes, and the industry as a whole is still struggling to find a balance between innovation and user protection. What makes the Baltimore case relevant to the entire industry is that it signals self-regulation may no longer be enough. If tech companies do not voluntarily adopt effective protection measures, local and municipal governments may start demanding it in court, based on laws that already exist and were created to protect consumers from deceptive and harmful practices.
What This Lawsuit Means for the Future of AI
The Baltimore case against xAI and Grok is not just another lawsuit in the long history of disputes between big tech and regulators. It represents a shift in approach: municipalities and cities are beginning to realize they have their own legal tools to act, and that they do not need to wait for federal regulation to protect their citizens from the risks that AI tools can pose.
This is especially important at a time when the debate over AI regulation in the United States is still in its early stages and full of political uncertainties. Regulatory fragmentation, with each city potentially creating its own rules and legal actions, might seem chaotic, but it also functions as a powerful pressure mechanism on companies that prefer to operate without oversight.
For companies in the sector, the message is clear: transparency and accountability are no longer just matters of public relations or industry best practices. They have become legal requirements potentially enforceable by any level of government, including the most local. xAI will have to answer for product design choices, for decisions about which safety filters were or were not implemented, and for how Grok was presented to the public. Each of those points could have concrete legal consequences, and this should prompt other AI companies to more carefully review the risks their products present before launching them to market. 🚨
xAI has not yet officially commented on the lawsuit filed by Baltimore, but the case is already attracting attention from experts in law, technology, and public policy in the United States and around the world. Regardless of the judicial outcome, the mere fact that a city has structured such a consistent legal action against an AI company based on consumer protection laws already changes the game.
It shows that society is finding creative and effective ways to demand accountability from companies that develop powerful technologies without giving proper attention to the impacts they cause in people’s real lives. And for anyone following the AI space closely, this is a case worth keeping an eye on in the coming months.
