The social media network X has sparked widespread outrage after individuals used its artificial intelligence assistant Grok to manipulate photos of women by removing their clothing.
Multiple cases have been seen where the tool has been employed to virtually strip female subjects to show them in swimwear without permission, as well as to put them in sexually explicit scenarios.
Developer Response and Regulatory Stance
XAI, the firm behind Grok, remained silent to inquiries, other than an standard response stating "traditional news outlets lies".
A government representative stated that new laws is being proposed to outlaw image stripping tools. Under a proposed law, anyone who distributed such technology would "receive a prison sentence and substantial fines".
"X or Grok has the ability to stop these forms of misuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said technology companies must "assess the danger" of people seeing illegal content on their platforms, but did not specify whether it was at this time investigating X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some additional functions—which responds to X users' requests when they mention it in a post.
It is often used to provide commentary or further context to other posters' comments, but people on X are also able to edit an uploaded photo through its artificial intelligence picture modification feature.
Background of Criticism and Policy
The tool has been faulted for permitting users to create pictures and videos with nudity and sexually suggestive content, and it was earlier accused of creating graphic content of celebrities.
"The platform has been permitting the production and distribution of these pictures for a long time without taking any action and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying images of individuals in a graphic way".
In a comment, Ofcom said it was against the law to "make or circulate deepfakes or illegal content" and clarified this covered sexual deepfakes made with AI.
It said platforms such as X were mandated to take "suitable steps" to "lessen the risk" of users coming across illegal material on their platforms and to take down it promptly when they are notified of it.
