EU Plans Ban on AI Chatbots Simulating Emotions for Minors

The EU plans to ban chatbots from simulating emotions or interpersonal relationships in interactions with anyone under the age of 18 as the bloc tries to stop children from developing emotional dependencies on the technology.

The rules will be part of the EU Kids Act, proposed on Thursday, which includes similar restrictions on minors’ access to social media and video-sharing platforms. Children under 13 will be banned from accessing chatbots without their guardians’ supervision, and digital platforms will have to build child-safety protections into their products or face fines of up to 6% of total worldwide turnover, according to a draft of the legislation seen by WIRED.

Chatbots operating in the EU will also not be allowed to retain memory of interactions between separate conversations under the new rules, since the Commission worries this could lead to systems accumulating sensitive data of minors over time and potentially reinforcing harmful interaction patterns.

Providers of online social networking sites, video-sharing platforms, and games should not automatically activate or encourage the use of AI chatbots on their services, the proposed law will also stipulate. That includes not pinning AI companions and general conversational chatbots at the top of a minor’s contacts list, it says—as had been done by Snapchat.

The proposal is limited to AI companions and general conversational chatbots seen by the Commission to pose serious risks to minors health, safety, and well-being. Simeon de Brouwer, a policy adviser at the Brussels-based advocacy group European Digital Rights, welcomes the chatbot measures, but tells WIRED: “it doesn’t make sense to restrict them to young people only.” “Don’t we all deserve to be protected from AIs designed to be addictive, from AIs which create emotional dependencies, or from AI features which are activated and pushed by default onto users?” The EU says its Digital Fairness Act, due to be unveiled later this year, is expected to propose clamp downs on addictive design more widely.

In documents seen by WIRED the European Commission says it is worried that dependency on chatbots may negatively impact a child’s development and its relationships with other humans. It’s also concerned by chatbots giving minors dangerous advice, including on their mental health and personal development, which it said can shape and encourage risky behaviour.

“Platforms have been playing fast and loose with children’s mental health,” President Ursula von der Leyen said in a press conference on Thursday. Around half of Europeans aged 11 to 25 have discussed intimate or personal matters with AI chatbots, according to an Ipsos BVA survey across France, Germany, Sweden and Ireland. Australia and the UK are both attempting to restrict minors’ access to chatbots through their social media bans.

It also highlighted the harms of deepfakes generated by chatbots, which von der Leyen nodded at in Wednesday’s State of the Union speech: “We do not have to accept that girls have their photos used for AI-generated sexualised images … Europe has the power to act.” While the EU AI Act has required synthetic content to be labeled as AI-generated since August, the rules currently only apply to deepfakes generated for commercial use.

The Commission also blames digital distraction for a decline in basic skills in young people. Children who say they use AI chatbots for schoolwork get lower test scores than those who don’t, according to research spanning 91 countries by the Organisation for Economic Co-operation and Development. Concerns about cognitive impacts and deskilling “expose the absence of children’s rights from the design of GenAI’s tools,” researchers from independent network EU Kids Online wrote in a report.