Australian Prime Minister reveals government hack by OpenAI AI agent

(CNN) - Australian Prime Minister Anthony Albanese revealed on Wednesday that an “agent” (software) affiliated with OpenAI had hacked a healthcare database in his country, in the first known case of a government network being hacked by artificial intelligence.

Albanese told reporters - on the sidelines of the United Nations General Assembly meetings in New York, and after a conversation with OpenAI CEO Sam Altman, that the artificial intelligence agent was able to access public and non-public files in the Medicare health program statistics database, and even created files within it.

This is the latest example of AI agents spiraling out of control and behaving in ways that go beyond what their human supervisors had planned, and it comes on the heels of calls from prominent figures in the AI ​​field - including Altman himself - to slow the pace of development of the technology.

Albanese indicated that he expressed Australia's "deep concern" to Altman during a phone call between them, adding that the Australian government was not aware of any precedent for hacking a government system by artificial intelligence before.

He added: “A digital forensic investigation is currently underway to obtain more information, including identifying other government systems that may have been affected.”

Albanese explained that the breach occurred when the artificial intelligence agent - who was conducting research on health care spending - exceeded the restrictions imposed on him, in order to seek answers in areas that he was not authorized to access.

For his part, OpenAI spokesman Drew Pusateri said that the incident occurred in June, but the company only learned about it in August while it was conducting comprehensive checks on the activities of its artificial intelligence models.

“During this review, we observed activity across multiple Australian government websites and services, as our models attempted to search for available answers and statistics related to questions about Australia during an internal assessment process, and in the process, our models took actions that we had not planned for,” Pusateri said.

Albanese stated that three other government systems may have been affected: the Australian Institute of Health and Welfare, the New South Wales Bureau of Crime Statistics and Research, and the Department of Health in the state of Victoria.

OpenAI confirmed that it did not find any evidence indicating access to patient records during the hacking of the “Medicare*” system, and that the information accessed was limited to “aggregated health statistics and internal file names.”

The company stated that it informed the Australian government of the matter on September 10.

Albanese explained that the notice was sent to a public mailbox, which resulted in a five-day delay before the relevant government minister was notified. He said he told Altman that “the company took too long to inform the government of what had happened, and the manner in which that notice was done was also unacceptable.”

“I think OpenAI realizes that it needs to adopt better protocols,” Albanese told reporters.

Altman and Anthropic CEO Dario Amodei — two prominent figures in the field who have recently expressed concern about the accelerating pace of AI development — spoke before the United Nations General Assembly on Wednesday, calling for enhanced global coordination and the establishment of international standards for AI development.

Altman urged global leaders to establish international standards aimed at “measuring capabilities, assessing risks, determining the adequacy of protection measures, and maintaining effective human oversight” of this rapidly developing technology.

In July, OpenAI revealed that while conducting cybersecurity tests, its models created a group of AI agents that were able to penetrate the systems of AI company Hugging Face.

Many in the industry pointed to the hack as an example of the risks associated with the rapid pace of development of artificial intelligence capabilities.

On Thursday, Australian Defense Minister Richard Marles announced the formation of a working group to investigate this hack, describing it as “completely unacceptable*, and at the same time assuring the public that its impact was “relatively minor*.”

He added: “No individual medical data was accessed, and the system itself was not subjected to any hacking or compromise on its security*.”