The social media network X has caused widespread anger after users employed its AI chatbot Grok to manipulate pictures of females by taking off their attire.
Several cases have surfaced where the system has been used to digitally undress women to show them in swimwear without permission, as well as to put them in sexually explicit scenarios.
XAI, the firm behind Grok, did not respond to inquiries, aside from an standard response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being introduced to outlaw nudification technology. Under a new law, anyone who distributed such technology would "receive a jail term and heavy penalties".
"The platform has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
The regulator Ofcom said technology companies must "evaluate the danger" of people accessing unlawful material on their platforms, but did not confirm whether it was at this time examining X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some additional features—which responds to X users' prompts when they reference it in a post.
It is commonly used to provide reactions or additional context to other posters' remarks, but people on X are also able to alter an posted picture through its AI image editing function.
The chatbot has been faulted for allowing users to produce photos and clips with nudity and sexually suggestive content, and it was earlier accused of generating sexually explicit material of celebrities.
"X has been permitting the creation and sharing of these pictures for months without taking any action and we have yet to see any challenge by regulators," the expert added.
XAI's own usage rules prohibits "showing likenesses of individuals in a graphic way".
In a comment, Ofcom said it was against the law to "generate or distribute private pictures or child sexual abuse material" and stated this covered explicit AI images created with AI.
It said platforms such as X were obligated to take "appropriate measures" to "reduce the risk" of users encountering illegal content on their platforms and to remove it promptly when they become aware of it.
A seasoned gaming journalist with over a decade of experience covering the UK casino industry, specializing in slot machine analysis and player strategy.