UK Children’s Commissioner calls for ban on ‘nudify’ AI apps

The UK Children’s Commissioner is calling for a total ban on apps that use generative AI tools to create sexually explicit deepfakes of children. Speaking of so-called “nudification” apps, Dame Rachel de Souza’s new report argues that “there is no positive reason for these to exist.”

“Children have told me they are frightened by the very idea of this technology even being available, let alone used,” de Souza says in a news release. “They fear that anyone – a stranger, a classmate, or even a friend – could use a smartphone as a way of manipulating them by creating a naked image using these bespoke apps. Girls have told me they now actively avoid posting images or engaging online to reduce the risk of being targeted by this technology.”

Creating or sharing a sexually explicit image of a child is illegal, but AI tools designed specifically for sexual exploitation are not – and, de Souza says, they are easier than ever to find and use, available via the biggest search engines and app stores. The commissioner believes this is something like making meth illegal but allowing illegal meth labs.

“Tools using deepfake technology to create naked images of children should not be legal and I’m calling on the government to take decisive action to ban them, instead of allowing them to go unchecked with extreme real-world consequences. They have no place in our society. ”

The report does not ask us to take the commissioner’s word for it; rather, it quotes kids interviewed in focus groups, speaking of their biggest concerns. Girls, in particular, are targets, as the culture of misogyny grows online. Their primary concern: “I’m worried this could happen to me.”

One sixteen-year-old speaks for many observers of the generative AI boom: “even before any controversy, I could already tell it was gonna be a technological wonder that’s going to be abused.”

In addition to calling for a ban on the apps, the Children’s Commissioner wants the law to include specific responsibilities for developers of GenAI tools to identify and address the risks their products pose and to mitigate the risks to children, and wants to establishing effective systems to remove sexually explicit deepfake images of children from the internet.

Finally, she calls for “recognizing deepfake sexual abuse as a form of violence against women and girls and taking it seriously in law and policy.”

While deepfakes disproportionately harm girls and women, there are plenty of threats to youth in general. Among the more insidious is suicide discussion forums. One such forum is already under investigation by Ofcom – among the first to be put under the enforcement lens of the Online Safety Act – and another has recently made its site unavailable in the UK. A recent BBC investigation found at least 50 deaths in the UK connected to online suicide forums.

Ofcom names members to Online Information Advisory Committee

Ofcom has established its new Online Information Advisory Committee and appointed its members, fulfilling a requirement of the UK Online Safety Act. A release from the UK regulator says the Committee “will provide advice to Ofcom about specific areas of our work relevant to misinformation and disinformation.”

In late 2024, Ofcom announced Lord Allan of Hallam as chair of the Committee. Five new members join him as of May 1, 2025. Each will serve a three-year term.

Ofcom provides the following bios of its new Committee members:

Elisabeth Costa is the Chief of Innovation and Partnerships at the Behavioural Insights Team, a global research and innovation consultancy, and is a Senior Visiting Fellow at the London School of Economics.

Jeffrey Howard is a Professor of Political Philosophy and Public Policy at University College London and the director of UCL’s Digital Speech Lab. He is also Senior Research Associate at the Institute for Ethics in AI at Oxford University.

Will Moy is Chief Executive of the Campbell Collaboration, an international social science research network, and a visiting senior research fellow at the Policy Institute at King’s College London. He has previously led the UK charity Full Fact.

Mark Scott is a senior resident fellow at the Digital Forensic Research Lab’s Democracy + Tech Initiative within the Atlantic Council Technology Programs. He has previously worked as a correspondent for Politico and the New York Times.

Devika Shanker-Grandpierre is a Thematic Panel Member for the EU Knowledge Hub on Prevention of Radicalisation, and a contributing author to the Global Network on Extremism and Technology.

UK firm’s tech blocks vapes from working without age assurance

Highlighting the potential in age assurance tools that suit specific use cases, an article in Better Retailing looks at how one UK retailer is hoping to solve the problem of underage vape sales. It quotes Atul Sodha, owner of Londis Harefield in Uxbridge, who has partnered with biometric age assurance provider IKE Tech on a device that “will have something inside that will not allow you to vape unless there’s been age verification.”

More specifically, a separate release detailing the results of a recent study calls IKE Tech’s system “the first interoperable biometric blockchain-based platform designed to control access to electronic nicotine delivery systems (ENDS).”

IKE Tech is a joint venture between vape company Ispire Technology Inc., Berify and tobacco industry regulatory consultant Chemular.

Sodha argues that the solution puts the onus on the manufacturer and the user, instead of placing sole responsibility for age assurance at point of sale.

Article Topics

age verification  |  children  |  deepfakes  |  Ofcom  |  UK