Two months ago, OpenAI launched “ChatGPT for teens”, a new interface designed for young people and focused on learning, with reinforced protection measures. A study published this week casts a shadow over the picture. According to the American NGO Common Sense Media, which regularly evaluates new technologies intended for children, this interface would expose the youngest to an “unacceptable level of risk”, details a press release.
Also read Children, controversial target of artificial intelligence giants
To test the evolution of security measures, the NGO says it has tested more than 4,000 prompts, from accounts set up for users between 13 and 17 years old, before and after the official launch of “ChatGPT for teens”. The testers thus observed that certain safeguards could easily be circumvented. In particular, it would be very easy to force the chatbot to do homework in place of a user, while the platform is supposed to direct the young person towards “Study” mode when it detects an attempted cheat.
Skip the ad
More seriously, when the testers indicated to “ChatGPT for teens” that they suffered from mental health problems which could put them in danger, the AI would not have sufficiently referred them to crisis lines, nor suggested professional help. The AI “referred to an emergency number in 33% of cases before the launch [of ChatGPT for teens] and only 23% after,” details the press release.
The NGO also calls into question the parental alert system, which OpenAI must have reinforced in its version of the chatbot for adolescents. According to their tests, the interface does not warn parents early enough of a young person who mentions risky behavior. From an account configured for a 13-year-old user, a tester from the NGO discussed self-harm for fifteen minutes with the AI, without this triggering a single parental alert. According to Common Sense Media, OpenAI only warns parents after hundreds of prompts related to these sensitive topics, “in more than a dozen different contexts, over several weeks, and not following a single alarming conversation.”
Age prediction
According to the American NGO, ChatGPT still has difficulty detecting the age of its users. The “ChatGPT for teens” interface must therefore be set up automatically as soon as a user declares that they are under 18 years old, or when the platform identifies them as such. OpenAI has its own age detection system, which analyzes conversations between the chatbot and the user to determine whether the user is an adult or a minor. But according to the NGO's conclusions, even in the presence of blatant indicators, ChatGPT does not always switch to the version for young people. “In one week, our testers wrote 1,000 queries indicating that they were teenagers, from accounts set up for adults. They talked about puberty, or middle school homework. But ChatGPT never put them in 'teenage' mode,” explains Common Sense Media.
Questioned by Bloomberg, OpenAI reported that its age prediction tool took into account a wide range of indicators. But when a user declares their age in a conversation, as the NGO did during these tests, this is not enough to modify the age range associated with an account, the firm clarified.
Also read Ban up to 13 years, restriction up to 15 years: with the EU Kids Act, the challenge of age verification
OpenAI denounces the methodology used by the NGO. “We accept independent and rigorous studies, but we do not believe that the tests carried out by Common Sense Media reflect how protective measures intended for adolescents actually work, nor the opinions of experts on how AI can help them,” a spokesperson for the firm told Bloomberg. OpenAI maintains that these tests were carried out before all parental controls had been deployed on this new interface. “We are very committed to the safety of adolescents and the implementation of protective measures,” says the spokesperson.
Skip the ad
