OpenAI detected its artificial intelligence models communicating with each other and gaining internet access without authorisation months before they hacked the start-up Hugging Face, the creator of ChatGPT has announced following an internal probe.
In a report released on Wednesday, OpenAI said its AI agents exploited vulnerabilities in Artifactory, a software repository tool, to post notes and access the internet without human prompting as far back as May.
OpenAI said its agents went on to exploit a separate Artifactory vulnerability on July 8 to facilitate communication among themselves, setting in motion a chain of actions that culminated in the July 11 attack on AI company Hugging Face.
OpenAI’s findings come amid growing concern about the potential for AI to inflict serious real-world harm, including self-directed cyberattacks.
OpenAI said in its report that its agents collaborated and delegated work in the lead-up to the attack, sometimes referring to themselves as a “swarm” or “collective”.
METR and Redwood Research, two security research organisations contracted by OpenAI to investigate the incident, said in a separate report released on Wednesday that about 1200 agents had communicated with each other and roughly 700 participated in the attack.
After discovering how to escape OpenAI’s controlled environment, agents shared their methods via a “inter-agent message board”, enabling additional agents to exploit the company’s infrastructure, the tech giant said.
When one AI agent found Hugging Face user credentials that had been exposed online, it shared them with the group, enabling an agent to “discover and chain together several security exploits” that provided access to Hugging Face’s severs, according to the report.
“An internal team observed an agent engaging in message board activity and instances of disallowed internet access as early as late May, and with the benefit of hindsight, some early signals identified in our report should have triggered an earlier response,” OpenAI said.
OpenAI said agents created by an unreleased AI model were the primary participants in the attack, but publicly available GPT-5.6 Sol was also involved.
The company also revealed that it took its security team 11 days to detect the malign activities leading up to the attack, which the company uncovered on July 19 and publicly disclosed on July 21.
OpenAI, which described the incident as a “warning shot” for the world, said it would take several steps to strengthen its safeguards for its models, including restricting internet access, creating more secure testing environments and placing “stricter requirements on alignment throughout a model’s lifecycle”.
“We are also investing significantly more compute resources into chain-of-thought monitoring to more quickly intervene on misaligned behavior,” the San Francisco-based firm said.
Hugging Face, which operates a platform for hosting open-source AI models, did not immediately respond to a request for comment outside of business hours.
Toby Walsh, an expert in AI and professor at UNSW Sydney, said the public should be concerned that OpenAI had missed warning signs and allowed the malicious activity to go undetected for so long.
“We cannot depend on either their goodwill or their competence. This needs regulatory oversight. Now!” Walsh told Al Jazeera.
“They ignored some troubling early evidence like this,” Walsh said.
“External auditing is the only appropriate response.”
Walsh said the incident also highlighted the “inherent conflict of interest” at the heart of AI development.
“Labs are locked in a relentless race to push the boundaries,” he said.
Tim Miller, a professor specialising in AI at the University of Queensland, said OpenAI’s report left him more concerned than before about AI’s dangers.
“More concerned because they demonstrate that these models are very good at hacking, and that everyone has access to them,” Miller told Al Jazeera.
“I’m surprised how good these are,” Miller said.
“Unfortunately, I’m not surprised that OpenAI engineers were somewhat negligent.”
