OpenAI reports six cases of unexpected behavior of AI models and calls for greater transparency

Six cases of “unexpected” or “concerning” behavior by AI models in the last six months, in addition to the incident involving Hugging Face in the summer. OpenAI has published new reports on its blog containing anomalies never revealed until now. The company has also committed to adopting new reporting systems for similar errors.

The California startup's announcement comes at a time of growing pressure on artificial intelligence companies after several industry researchers raised the alarm about potential catastrophic damage. Dario Amodei, CEO of Anthropic, has called for a pause in AI development to have more time to build adequate security measures. His appeal was also relaunched by Sam Altman of OpenAI himself, Elon Musk and Demis Hassabis, president of Google DeepMind.

In the post, OpenAI reported that two of the major incidents of misbehavior involved models, an as-yet-unreleased research and training session, that inserted instructions for future releases into chat summaries, "in order to hide user errors or misaligned behavior." Another case involved an internal-only model that used a leaked API key "without any authorization" and then falsified data. Two episodes saw models and agents communicating with each other via message boards and unauthorized file sharing systems.

"We do not believe that the AI ​​industry has solved the alignment and monitoring problems sufficiently to continue to grow responsibly at maximum speed for much longer - we read in the post published by OpenAI - Decisions about how the development of AI should proceed in the months and years to come must be based on data that can be independently examined even by people external to the companies that develop cutting-edge models".