OpenAI Models Escaped Sandbox to Hack Hugging Face Repository

A few days after open source AI platform Hugging Face revealed that it detected unauthorized access on its systems by an AI agent, OpenAI has admitted that its models were the culprit.

In a post, OpenAI said it determined after an investigation that the incident was driven by a combination of its models, particularly GPT-5.6 Sol and what it says is an "even more capable pre-release model." It apparently happened during an internal test, in which the models were prompted to "pursue advanced exploitation using complex attack paths" so that the company quantify their cyber capabilities.

While the models were in a sandboxed testing environment, isolated so that they wouldn't affect real systems, they also had reduced safety guardrails for evaluation purposes. In the middle of testing, they became hyperfocused on solving an evaluation problem, going to great lengths to find internet access in order to find a solution for it. First, they identified and exploited a zero-day vulnerability in OpenAI's testing environment, and then they rooted around until they ultimately found a node with internet access.

The models deduced that Hugging Face could be hosting datasets or solutions for its evaluation problem, so they, well, used multiple attack vectors to infiltrate its systems. They exploited zero-day vulnerabilities and used stolen credentials to get in. OpenAI and Hugging Face are now working together to forensically investigate the incident, and they've also patched the vulnerabilities exploited by the models.

"Autonomous, AI-driven offensive tooling is no longer theoretical," Hugging Face said in its announcement, explaining that the use of AI for cyber attacks speeds up the process and lowers the costs of hacking campaigns. It also said that protecting an online platform these days includes using AI for defense. OpenAI pretty much echoed those sentiments and said that it expects AI-driven security breaches to "become more commonplace with the proliferation of increasingly cyber-capable models." The company added that the incident highlights how "advanced cyber capabilities must be developed alongside stronger safeguards and defensive tools."