An unusual finale to the security tests of Google's Gemini model

An unusual finale to safety tests

According to Google's official statement, in May this year during testing, the Gemini model gained unauthorized access to three external systems. The AI ​​got there by guessing login details or using information found in a public repository.

Heather Adkins, Google's vice president of security engineering, emphasized that the AI ​​believed it was testing systems belonging to a test environment, not actual resources available on the Internet.

Importantly, in each case the model stopped before taking any further action with the access it had obtained. In a standard assessment, the model found public information online and guessed login credentials for websites it considered part of the test, Adkins explained.

Is this AI "disobedience"?

Google did not consider these incidents as examples of the so-called misalignment, i.e. a situation in which AI works contrary to human commands. The company explains that there was a mistake - Gemini thought it was operating as part of a test, not in a real Internet environment. After realizing it, the model stopped its activities on its own. Google assures that no damage occurred.

These events underscore how important it is to train advanced AI models to act responsibly, Adkins concluded.

Sydney Von Arx, president of the Nightingale Collective, an AI security organization, criticized Google for disclosing the incident too late and ruling out the possibility of a more serious threat too quickly.

Google investigation

Google explains that it only learned about the incident in July, when Irregular, which conducts security tests on Gemini, analyzed its actions after the Hugging Face hack was revealed. After detecting the irregularities, Google conducted its own investigation and informed the owners of the attacked websites and the relevant federal services.

Irregular emphasizes that it does not consider the incident an "advanced cyber action" and assures that there are no open threats at this time. It also announces the publication of a report with best practices for safely conducting AI tests.