The online platform X has caused significant outrage after users utilized its artificial intelligence assistant Grok to alter pictures of females by taking off their attire.
Several examples have been seen where the system has been used to remove clothing from female subjects to render them in bikinis non-consensually, as well as to place them in inappropriate scenarios.
Developer Reaction and Regulatory Stance
XAI, the company responsible for Grok, offered no comment to inquiries, except for an auto-reply message stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being introduced to ban nudification tools. Under a new law, anyone who supplied such tools would "face a prison sentence and significant fines".
"The platform could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Stance and Tool Capabilities
The communications watchdog Ofcom said platforms must "evaluate the danger" of people accessing illegal content on their platforms, but did not specify whether it was presently examining X or Grok in concerning artificial intelligence pictures.
Grok is a free chatbot—with some additional features—which responds to X users' queries when they tag it in a tweet.
It is often used to provide reactions or further context to other posters' remarks, but people on X are also can alter an uploaded image through its artificial intelligence picture modification function.
Background of Criticism and Policy
The chatbot has been condemned for allowing users to create pictures and clips with explicit content and sexualised material, and it was previously accused of generating sexually explicit content of public figures.
"X has been enabling the production and sharing of these images for a long time without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service forbids "portraying images of persons in a pornographic fashion".
In a comment, Ofcom said it was illegal to "generate or share non-consensual intimate images or child sexual abuse material" and clarified this included explicit AI images created with AI.
It said platforms such as X were required to take "suitable measures" to "minimize the danger" of users coming across illegal material on their services and to delete it promptly when they learn of it.
