EU to vote on banning sexualized deepfakes after X’s Grok backlash

EU lawmakers will vote on a ban on AI systems generating sexualized deepfakes this Wednesday, following outrage over images that feature digitally undressed women, including minors, created by X’s AI chatbot Grok.

On Friday, the Council of EU reached an agreement on “prohibiting AI practices regarding the generation of non-consensual sexual and intimate content or child sexual abuse material,” the Cypriot Presidency of the EU Council announced.

“Systems capable of generating, manipulating or reproducing such material pose a severe risk to victims’ human dignity, personal autonomy, integrity and private life, with potentially serious lasting psychological and other harms and abuse at scale,” the new proposal notes.

​The ban is expected to become law after the European Parliament and member states negotiate on the final text as part of the amendments to the EU AI Act. The amendments are part of the so-called Omnibus legislative package in the EU’s simplification agenda.

The sexual deepfakes ban was included in the Omnibus after a backlash from European governments over Grok’s production of millions of sexualised deepfakes.

In January, Elon Musk’s X was placed under investigation by the European Commission, which is set to determine whether the social media platform violated the bloc’s Digital Services Act (DSA). The company, which was already fined 120 million euros (US$137.7 million) for breaching transparency obligations under DSA, may face fines of up to 6 percent of its global annual turnover.

The investigation will be aided by Ireland’s data protection regulator, which launched a probe into X to examine the firm’s processing of personal data under the EU’s GDPR. Other EU member states have launched their own inquiries, including France and Spain.

On January 14th, X responded to the controversy by stating it had a “zero-tolerance” policy towards sexualised deepfakes of children and women, and pledged to introduce measures that would end the production of such images. Users, however, could still bypass the restrictions on January 20th,  according to research from non-profit AI Forensics.

Article Topics

AI Act  |  deepfakes  |  EU  |  Europe  |  generative AI  |  Grok  |  regulation