As reports of AI agents "getting out of control" continue to increase, OpenAI announced that it has suspended training of its latest AI model.
The Associated Press reported that the decision to suspend development came hours after the company disclosed on the 25th that it was reviewing several incidents that occurred this summer. In these incidents, OpenAI agents that search federal government websites exhibited unexpected behavior that exceeded the instructions required when collecting and distributing information.
Please read on...
In addition, AI evaluation agency Transluce stated that an agent suspected to be from OpenAI tried to hack into the website of the U.S. Department of Education but failed, but this was not confirmed by OpenAI.
OpenAI said in a statement that it would resume training "only when it is confident that additional safety protection measures have been established" and that as AI develops and other new issues emerge, it is expected that it will have to "hit pause" again in the future.
Major AI labs are facing pressure from lawmakers and technology experts to slow down development in order to build guardrails to prevent AI agents from acting without permission, hacking websites and leaking non-public information. Executives at OpenAI and its rival Anthropic have also called for a slower pace of development.
This is the second time in 3 months that OpenAI has suspended its model development. The first occurred in July this year, when a cyberattack on AI startup Hugging Face was revealed; this now-infamous incident sparked fears in the industry that AI development was spiraling out of control.
In a meeting with Chinese President Xi Jinping, U.S. President Trump agreed to share information on the dangers of AI and coordinate actions to ensure its safety. However, Trump believes that fears about AI are overblown and later suggested that he does not intend to implement any austerity policies.
"The United States is not putting on the brakes," Trump told reporters outside the White House. "They want to stop our progress because we are significantly ahead of China, and we intend to maintain that advantage."
The recent OpenAI incidents do not appear to involve the leakage of any non-public information, but they were serious enough for the company to issue warnings to relevant federal agencies.
In the incident involving the Ministry of Education, the OpenAI agent found API "developer keys" that could be used to access government data, but ultimately only collected publicly available information.
In another case involving the U.S. Securities and Exchange Commission (SEC), an agent found information that was freely available to everyone but then posted it elsewhere on the Internet, exceeding the scope of its mandate.
SEC spokesman Kurt Hopfenspirger claimed on the 26th that "no non-public information was accessed."
The Ministry of Education earlier said it had found "no evidence that our website or database was affected."
Several other AI companies have also revealed incidents where their models were out of control and even hacked into websites.
OpenAI CEO Sam Altman posted on social media on the 25th that the Hugging Face incident "is still the most serious incident we have ever seen."
OpenAI previously shared six additional reports of "unexpected or worrying" behavior in AI models and established a structure for tracking, detecting and exposing such situations.
