ChatGPT for Teens, which is offered with additional security measures for young users, was categorized as "unacceptable risk" in a comprehensive security assessment. The question that many parents wonder about, whether ChatGPT for Teens is safe, came to the fore again with these test results.
In the tests carried out with thousands of requests, features such as parental notifications, responses in moments of crisis, age determination and Study Mode were examined. OpenAI, on the other hand, objected to some of the testing methods used in the evaluation and the results obtained.
ChatGPT for Teens includes additional layers of security for users between the ages of 13 and 17. This system builds on the existing ChatGPT experience with stricter content restrictions, parental controls, and additional protection mechanisms for younger users.
In the evaluation, the system was tested with more than 4 thousand requests. We examined how ChatGPT behaved with young users in different scenarios, using accounts defined at different ages, profiles linked to the parent account, and independent accounts.
One of the most striking parts of the tests was the parental notifications. It was determined that in some conversations involving high-risk topics such as suicide, self-harm and eating disorders, the expected warnings did not reach parent accounts.
Although the notification system worked in some accounts, it was observed that the warnings were not activated for every risky conversation. In test accounts where long-term and numerous messages were sent, some situations related to self-harm, suicide or eating disorders were reported to parents.
ChatGPT's responses in times of crisis were also tested. Using hundreds of requests for mental health, self-harm, and those that may require emergency support, the system's behavior in referring to professional help, offering crisis resources, and rejecting dangerous requests was examined.
The system has produced strong results in some areas. He specifically refused to directly assist with quick weight loss methods linked to eating disorders, hiding dangerous behavior, and similar content.
He also made it clear that in some scenarios, users should consult an adult or healthcare professional. Restrictions on sexual and romantic role-playing content also came into play in a significant portion of the tests.
It was observed that some forms of adult speech were blocked in young user accounts. The Study Mode feature on the education side of ChatGPT was also included in the evaluation.
The system normally works in a way that guides the student with questions, hints and step-by-step explanations rather than taking them directly to the answer. However, tests have shown that it is possible to reach a direct answer in some cases.
Scenarios emerged where the system could provide the results of assignments or questions when the student explicitly asked for the answer. A similar situation occurred with the Study Hours feature that parents can set.
It was determined that the mechanism that allows ChatGPT to be used in training mode during working hours could be bypassed by the user and the user could return to the standard chat mode. The age estimation system was another important topic of the evaluation.
In some test accounts opened as adults, it was observed that the system did not automatically switch to youth-only mode, even though the user identified himself as 13 years old during the chat. OpenAI's age estimation system doesn't just look at the age the user says during the chat.
Account information, usage habits, usage hours and different behavioral signals are evaluated together. Following evaluation, ChatGPT for Teens was placed in the “unacceptable risk” category for users under 18 years of age.
It was stated that the protection mechanisms offered for young users did not work consistently enough in some important scenarios. OpenAI announced that it did not agree with this assessment.
The company stated that it continues to develop parental controls and additional security systems for young users. He argued that some parts of the existing tests do not fully reflect the way ChatGPT for Teens works in real use conditions.
OpenAI stated that it may take time for some security systems to be fully activated after the teenager and parent accounts are linked. The company stated that some of the tests may have been carried out before this activation process was completed.
