🔗 Share this article

Tech firms and child protection organizations will be granted permission to evaluate whether artificial intelligence tools can generate child abuse material under recently introduced British legislation.

Substantial Rise in AI-Generated Harmful Material

The announcement came as findings from a safety watchdog showing that cases of AI-generated child sexual abuse material have increased dramatically in the past year, rising from 199 in 2024 to 426 in 2025.

New Regulatory Structure

Under the amendments, the government will permit approved AI companies and child safety organizations to inspect AI systems – the underlying systems for chatbots and visual AI tools – and verify they have sufficient protective measures to stop them from producing depictions of child sexual abuse.

"Fundamentally about preventing abuse before it occurs," declared the minister for AI and online safety, noting: "Experts, under rigorous conditions, can now identify the danger in AI systems promptly."

Addressing Legal Challenges

The changes have been introduced because it is against the law to produce and possess CSAM, meaning that AI developers and others cannot generate such images as part of a evaluation regime. Previously, officials had to delay action until AI-generated CSAM was published online before dealing with it.

This law is aimed at averting that problem by helping to stop the creation of those materials at their origin.

Legislative Structure

The amendments are being introduced by the government as modifications to the criminal justice legislation, which is also establishing a prohibition on owning, creating or sharing AI models developed to generate exploitative content.

Practical Impact

This recently, the minister visited the London base of a children's helpline and heard a mock-up conversation to counsellors featuring a account of AI-based abuse. The interaction depicted a teenager seeking help after being blackmailed using a explicit AI-generated image of himself, constructed using AI.

"When I learn about young people experiencing extortion online, it is a cause of extreme frustration in me and rightful concern amongst families," he stated.

Alarming Data

A leading internet monitoring foundation stated that cases of AI-generated abuse content – such as webpages that may contain numerous images – had significantly increased so far this year.

Instances of category A material – the gravest form of exploitation – rose from 2,621 images or videos to 3,086.

• Female children were overwhelmingly targeted, making up 94% of illegal AI images in 2025
• Depictions of infants to two-year-olds increased from five in 2024 to 92 in 2025

Industry Response

The legislative amendment could "represent a vital step to guarantee AI tools are secure before they are released," commented the chief executive of the online safety foundation.

"Artificial intelligence systems have made it so victims can be victimised repeatedly with just a few clicks, providing criminals the capability to make possibly endless amounts of sophisticated, lifelike exploitative content," she continued. "Material which further exploits survivors' suffering, and makes young people, particularly girls, less safe both online and offline."

Support Interaction Information

Childline also published details of counselling interactions where AI has been mentioned. AI-related risks mentioned in the sessions comprise:

• Using AI to rate weight, body and looks
• AI assistants dissuading children from consulting trusted adults about harm
• Being bullied online with AI-generated material
• Digital extortion using AI-faked pictures

During April and September this year, Childline conducted 367 counselling sessions where AI, conversational AI and associated terms were discussed, significantly more as many as in the equivalent timeframe last year.

Fifty percent of the mentions of AI in the 2025 interactions were connected with psychological wellbeing and wellbeing, encompassing using chatbots for support and AI therapy apps.