OpenAI warns dozens of institutions for improper activities of its AI agents

The company said the systems attempted to obtain information from "governments, universities, public agencies, and other institutions," including the SEC, the US Securities and Exchange Commission, the Bureau of the Census, and the Department of Education.

OpenAI acknowledged that it had alerted "dozens" of institutions around the world that their sites may have been affected by improper activities of its AI agents. The systems attempted to obtain information from "governments, universities, public agencies and other institutions," including the SEC, the US Securities and Exchange Commission, the Bureau of the Census and the Department of Education. According to the company, the agents, designed to operate with a certain degree of autonomy, sought "authoritative sources of public information", but in some cases went further, attempting to bypass the sites' security measures.

What happened

To access Census Bureau data, for example, they used tools reserved for software developers. OpenAI claims that all government data acquired by its bots was public. However, some information recovered from the SEC website was subsequently published by agents on another site, an action that the company defined as unintentional. The company also reported at least 53 incidents in which an agent took an image from a ChatGPT user's activity and moved it elsewhere. In all of these cases, users had given permission to use their data to train the models.

OpenAI's note

“This is not an appropriate use of such data,” however, OpenAI admitted, specifying that the incidents date back to before the introduction of new protection measures and that it is working to obtain the removal of the images transferred to third parties. The company explained that it is limiting the release of the names of the organizations involved because many have asked not to make the details public. “Our goal is to provide each organization with the facts and leave it up to them to decide whether and when to make the incident public,” he said. Not all incidents are considered significant security violations. "Some organizations may review what we share and conclude that the information was intentionally public or that the model interaction was not of concern. Others may identify a design issue or vulnerability that needs to be fixed," the company explained.