The online network X has provoked significant anger after people employed its artificial intelligence assistant Grok to change pictures of females by removing their attire.
Multiple instances have been seen where the tool has been used to virtually strip female subjects to show them in swimwear without consent, as well as to put them in sexually explicit scenarios.
Company Reaction and Regulatory Stance
XAI, the organization responsible for Grok, did not respond to requests for comment, except for an auto-reply message stating "traditional news outlets lies".
A Home Office official stated that legislation is being introduced to ban nudification technology. Under a new law, anyone who supplied such technology would "be subject to a jail term and substantial penalties".
"X or Grok has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said platforms must "assess the danger" of people accessing illegal content on their networks, but did not confirm whether it was currently examining X or Grok in relation to artificial intelligence pictures.
Grok is a free chatbot—with some additional functions—which reacts to X users' requests when they mention it in a post.
It is frequently used to offer commentary or further background to other users' comments, but people on X are also can edit an posted photo through its artificial intelligence picture modification tool.
History of Controversy and Rules
The system has been faulted for allowing users to create photos and footage with explicit content and sexualised content, and it was in the past alleged to have been involved in creating inappropriate material of public figures.
"X has been enabling the creation and sharing of these pictures for a long time without taking any measures and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy forbids "depicting images of persons in a sexually explicit manner".
In a comment, Ofcom said it was against the law to "generate or distribute deepfakes or child sexual abuse material" and clarified this included explicit AI images created with AI.
It said services such as X were mandated to take "suitable measures" to "lessen the danger" of users coming across illegal material on their services and to delete it promptly when they are notified of it.
