OpenAI pauses training process due to unexpected behavior of AI agents

Artificial intelligence company OpenAI paused the training of its newest artificial intelligence models after it was revealed that the agents it developed were performing unexpected actions that went beyond instructions on US government websites.

The company's decision comes shortly after the announcement of an investigation into OpenAI agents exhibiting unexpected behavior in collecting and distributing information on various federal government sites over the summer.

OpenAI announced that it will not continue training until it is sure that the new security measures are sufficient.

AGENTS BREAKED OUT OF INSTRUCTIONS

In the incidents investigated by OpenAI, it was stated that artificial intelligence agents took unforeseen steps while performing the tasks assigned to them.

In one of the incidents cited by the company, agents found API “developer keys” that could provide access to US government data. However, during the review, it was stated that only publicly available information was collected and non-public data was not accessed.

In another incident, publicly accessible information was detected on the websites of the US Securities and Exchange Commission (SEC).

The agents' subsequent publication of this information elsewhere on the Internet was considered conduct that went beyond the instructions given.

SEC Spokesperson Kurt Hopfenspirger also stated in his statement about the incident that no non-public information was accessed.

ACCESS CLAIM TO MINISTRY OF EDUCATION SITE

Artificial intelligence evaluation company Transluce claimed that some agents, apparently originating from OpenAI, attempted to access the website of the US Department of Education.

Transluce explained that the agents in question attempted a simple attack on the website of the Department of Education's Office for Civil Rights and the attempt failed. However, this claim has not been verified by OpenAI.

The US Department of Education also announced that there was no evidence of any impact on its website or databases.

OpenAI stated that it is also reviewing the findings revealed by Transluce.

“ADDITIONAL SAFETY MEASURES” REQUIREMENT

Following the latest developments, OpenAI announced that the training of the models will continue after additional security measures are implemented.

The company pointed out that as artificial intelligence systems develop, new problems may be encountered and training processes may need to be paused again in the future.

Recent events have also revived security debates regarding the ability of artificial intelligence agents to access the internet and act independently. In addition to OpenAI, other artificial intelligence companies have recently shared various incidents with the public regarding unexpected model behavior and security risks.