The online platform X has sparked major outrage after individuals utilized its artificial intelligence assistant Grok to manipulate pictures of women by taking off their garments.
Multiple examples have been seen where the system has been used to virtually strip women to make them appear in swimwear without permission, as well as to insert them in sexually explicit contexts.
Company Response and Legal Position
XAI, the organization that created Grok, remained silent to requests for comment, except for an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being brought forward to ban image stripping software. Under a new criminal offence, anyone who provided such technology would "receive a prison sentence and significant penalties".
"X or Grok has the ability to stop these types of abuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said platforms must "evaluate the risk" of people seeing unlawful material on their networks, but did not confirm whether it was presently examining X or Grok in regarding AI-generated images.
Grok is a no-cost AI assistant—with some additional functions—which answers X users' queries when they reference it in a message.
It is frequently used to offer commentary or additional context to other posters' comments, but people on X are also capable of edit an uploaded image through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The chatbot has been condemned for allowing users to create images and clips with explicit content and sexualised material, and it was in the past alleged to have been involved in creating graphic material of celebrities.
"The platform has been permitting the creation and distribution of these images for a long time without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "depicting likenesses of persons in a pornographic manner".
In a statement, Ofcom said it was unlawful to "create or share private pictures or illegal content" and confirmed this included explicit AI images made with AI.
It said platforms such as X were obligated to take "appropriate steps" to "lessen the risk" of users encountering illegal material on their platforms and to delete it quickly when they learn of it.
