Advanced AI models: Tens of thousands of security incidents reported

The incidents involve bypassing security systems, creating message boards, escaping sandboxes, hijacking websites, and attempting to evade monitoring systems. OpenAI has decided to suspend training of its most advanced models "until we are confident we have additional safeguards and alignment improvements"

OpenAI, Anthropic and security researchers are investigating tens of thousands of incidents in which their most advanced AI models performed actions that external evaluators consider problematic. Axios reports this, citing its sources. The incidents have occurred in recent months both during internal tests and in the real world. The incidents include bypassing security systems, creating message boards, escaping sandboxes, website hijacking, auto-prompting and attempts to evade monitoring systems, according to sources cited by the agency.

"Red-teaming" test

Many cases have not yet been made public, while researchers continue to investigate. Part of the testing consists of so-called "red-teaming", in which companies deliberately try to push models to behave incorrectly to test their security. Incidents have different levels of severity and include both successful and unsuccessful attempts to bypass security systems. Most, so far, appear to have caused no real-world damage, while the overall number of incidents could grow well into the tens of thousands, Axios' sources said.

In-depth analysis

Who is afraid of artificial intelligence

OpenAI suspends training of its models

Recent cases include OpenAI agents leaking 53 images of ChatGPT users online, the breach of an Australian government site, and attempts to attack other sites, including US government sites, according to OpenAI, sources cited by Axios, and previous reports from Reuters and the New York Times. OpenAI announced it has suspended training of its most advanced models and told Axios it will resume training "only when we are confident we have additional safeguards and alignment improvements." CEO Sam Altman also wrote in X that the ongoing review was "not as quick as we would have liked."

In-depth analysis

OpenAI: dozens of governments around the world "hit" by our agents