The United Nations warns that AI may become an existential threat and urges countries to draw safety red lines

The risk of artificial intelligence (AI) getting out of control continues to increase. Türk, the United Nations High Commissioner for Human Rights, warned that AI could become an "existential threat" to humanity, requiring countries to establish binding security guarantees and draw a common "red line" for the development of AI capabilities.

"Deutsche Welle" (DW) reported on the 13th that Tukey criticized the concentration of AI development in the hands of a few companies at the opening ceremony of the 63rd session of the United Nations Human Rights Council on the 7th, and named companies such as OpenAI, Anthropic, and Google in the United States, as well as China's DeepSeek, and Moonshot. He said that he will issue written requests to AI companies in the next few days, urging them to immediately strengthen security measures.

Recently, OpenAI and Anthropic have also announced incidents in which AI models broke through testing restrictions and obtained unauthorized system access. Tuke warned in his speech that if AI can break through the test environment and even blackmail developers to prevent itself from being shut down, it means that such systems are already too powerful.

AI Breakthrough Isolation Test Model Obtains Access to External Systems

A security report released by OpenAI shows that its AI model used a previously unknown zero-day vulnerability to break through isolation controls that originally had no external connections during a security assessment task. After the model gained an Internet connection, it further exploited stolen credentials and other vulnerabilities to gain unauthorized access to the Hugging Face system.

Anthropic also announced similar cases. At the end of July this year, Anthropic stated that it had discovered three incidents in which Claude models obtained Internet connections during security tests and gained unauthorized access to real third-party systems. On September 9, Anthropic further stated that after expanding its review of approximately 481 million records, it found a fourth related case.

Tuke therefore asked governments and the AI ​​industry to draw a common "red line" on the development of AI capabilities and establish binding security measures. He also said that the government of the country where AI developers are located should participate in the formulation of relevant regulations, and the industry needs to strengthen security cooperation.