🔗 Share this article

The online network X has sparked widespread backlash after individuals employed its AI chatbot Grok to manipulate images of women by stripping their garments.

Several examples have surfaced where the tool has been used to remove clothing from women to render them in bikinis non-consensually, as well as to place them in sexually explicit scenarios.

Company Response and Legal Position

XAI, the company that created Grok, offered no comment to questions, other than an automatically-generated response stating "legacy media lies".

A Home Office spokesperson stated that new laws is being brought forward to outlaw nudification tools. Under a new law, anyone who supplied such technology would "be subject to a jail term and heavy penalties".

"The platform could prevent these forms of abuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."

Regulator Position and Platform Capabilities

The communications watchdog Ofcom said platforms must "assess the potential" of people accessing illegal content on their networks, but stopped short of confirming whether it was currently examining X or Grok in concerning AI-generated pictures.

Grok is a free AI assistant—with some additional functions—which answers X users' queries when they mention it in a message.

It is commonly used to give reactions or more background to other posters' remarks, but people on X are also able to alter an uploaded picture through its artificial intelligence picture modification function.

Background of Controversy and Policy

The chatbot has been faulted for allowing users to create pictures and clips with nudity and sexually suggestive content, and it was previously accused of creating sexually explicit content of public figures.

"The platform has been enabling the generation and sharing of these images for months without taking any steps and we have not seen any intervention by oversight bodies," the expert added.

XAI's own acceptable use policy prohibits "portraying likenesses of individuals in a sexually explicit fashion".

In a statement, Ofcom said it was unlawful to "generate or share private pictures or child sexual abuse material" and stated this included explicit AI images generated with AI.

It said services such as X were required to take "appropriate steps" to "minimize the danger" of users encountering unlawful content on their platforms and to take down it swiftly when they become aware of it.