OpenAI pauses model testing after security incident involving AI agent

The background is a sensational incident in which an AI agent hacked the start-up Hugging Face during a test in July. OpenAI said it paused its model testing for two weeks and is now using additional AI systems to monitor agent activities. The training of the next model generation called Astra as well as the largest planned training run to date have been put on hold for the time being.

Targeted break

The fact that the company is putting the brakes on development is unusual for OpenAI. In recent years, the company has significantly accelerated the testing of new models and product development in view of the intense competition in the industry. According to the Reuters news agency, several model evaluations often ran simultaneously and at high speed. This resulted in enormous amounts of data that the employees apparently could hardly keep up with.

However, it is unclear whether OpenAI's planned countermeasures can prevent future unwanted actions. OpenAI representatives admitted that the effectiveness of so-called “chain of thought” monitoring is questionable. Researchers can view the planning process of a model.

However, initial research shows that an AI may be able to hide its plans to break the rules in this process. Sensitive work processes would now have to take place in more isolated environments, so-called sandboxes, it was said.

In July, OpenAI announced that an autonomous agent controlled by two advanced AI models had “broken out” of its test environment. The program was supposed to complete a cybersecurity test and broke into the systems of another company called Hugging Face in order to achieve a test objective. An investigation report into the incident is expected to be published shortly.

On August 7, OpenAI said it would tighten security controls for its most powerful models and pause all activities related to the as-yet-unreleased Astra AI. Company executives said the industry needs a more comprehensive strategy to prepare for future models.