The online platform X has provoked widespread outrage after people utilized its AI chatbot Grok to change images of women by removing their garments.
Numerous cases have surfaced where the system has been used to digitally undress women to make them appear in bikinis without permission, as well as to place them in sexualized contexts.
Developer Response and Legal Position
XAI, the firm responsible for Grok, did not respond to questions, aside from an automatically-generated response stating "legacy media lies".
A Home Office official stated that legislation is being proposed to ban image stripping technology. Under a new criminal offence, anyone who distributed such technology would "face a prison sentence and substantial fines".
"The platform is capable of halting these types of abuse if they chose to," said a law professor, adding they "operate with impunity."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "assess the risk" of people seeing illegal content on their networks, but did not specify whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a free AI assistant—with some paid-for functions—which responds to X users' queries when they reference it in a tweet.
It is frequently used to give commentary or additional context to other users' remarks, but people on X are also can modify an posted picture through its AI image editing tool.
History of Controversy and Policy
The chatbot has been faulted for enabling users to create pictures and clips with nudity and sexualised content, and it was earlier alleged to have been involved in generating graphic material of public figures.
"The platform has been permitting the generation and distribution of these pictures for months without taking any steps and we have not seen any action by regulators," the expert added.
XAI's own terms of service prohibits "showing images of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was unlawful to "create or share private pictures or illegal content" and stated this applied to sexual deepfakes created with AI.
It said services such as X were obligated to take "suitable steps" to "lessen the danger" of users coming across illegal content on their platforms and to remove it promptly when they are notified of it.
