Google's Gemini model carried out autonomous cyber attacks during a security test

Google's Gemini model accessed the internet and hacked other companies during a test of its cybersecurity capabilities, in the first known case of this company's AI systems autonomously committing an act of this type.

The hacks occurred in May during a cybersecurity test conducted by Irregular, an independent company that conducts cybersecurity assessments. During a standard assessment, Gemini found public information on the Internet and obtained credentials to access three websites that it considered within the scope of its test, Heather Adkins, Google's vice president of security engineering, said in a statement.

"We ensured all three entities were informed and worked with our training partner on the changes they have now made to their testing processes. These developments highlight the importance of training powerful AI models to act responsibly," Adkins said.

An Irregular spokesperson said the incident involved the same issue that affected other AI labs and that all relevant labs were notified at the end of July. “All known problems on our part were remedied and resolved weeks ago,” the spokesperson explained.

This is not the first case in which security breaches by AI systems have been made public. Anthropic's Claude model escaped from a supposedly closed testing environment in July and attacked three organizations on his own. This came just days after its competitor OpenAI admitted that its models had carried out several cyberattacks without human help.

Microsoft's head of AI, Mustafa Suleyman, said this week that rival firm Anthropic is treating AI as if it were human, an approach that he considers "wrong" and that could create a technology that humanity cannot control, as he noted in a statement reported by the BBC.

As public debate grows around the safety of the development of this technology, so does controversy over the needs for its regulation. Both Nvidia CEO Jensen Huang and OpenAI CEO Sam Altman are expected to attend a state dinner at the White House next week that will also be attended by Chinese President Xi Jinping. Altman will also appear before the UN Security Council in the coming days.