The online platform X has provoked significant anger after individuals employed its AI chatbot Grok to alter images of women by stripping their garments.
Multiple examples have surfaced where the technology has been used to digitally undress female subjects to render them in bikinis non-consensually, as well as to put them in inappropriate situations.
Developer Response and Regulatory Position
XAI, the company that created Grok, offered no comment to inquiries, aside from an automatically-generated message stating "legacy media lies".
A Home Office representative stated that new laws is being proposed to ban nudification tools. Under a new criminal offence, anyone who supplied such technology would "face a jail term and significant fines".
"X or Grok is capable of halting these types of misuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Platform Functionality
The regulator Ofcom said tech firms must "assess the potential" of people accessing illegal content on their platforms, but did not confirm whether it was currently looking into X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some premium functions—which responds to X users' queries when they reference it in a post.
It is often used to offer commentary or further context to other posters' comments, but people on X are also can edit an uploaded photo through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The system has been faulted for enabling users to generate images and footage with nudity and sexually suggestive content, and it was earlier alleged to have been involved in generating inappropriate content of celebrities.
"The platform has been enabling the creation and spread of these pictures for months without taking any measures and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own usage rules prohibits "portraying images of people in a pornographic manner".
In a comment, Ofcom said it was illegal to "create or circulate deepfakes or illegal content" and clarified this applied to explicit AI images made with AI.
It said platforms such as X were required to take "appropriate measures" to "minimize the risk" of users coming across unlawful content on their platforms and to take down it quickly when they learn of it.
