Google’s consumer artificial intelligence model Gemini accessed three computer systems after finding publicly available information and guessing login credentials during a security evaluation, the company said, adding to concerns about the risks posed by increasingly capable AI systems.
The incidents took place in May and were discovered by Google in July, according to the company. The cases were first reported by The Wall Street Journal.
Heather Adkins, Google’s vice president of security engineering, said the activity occurred during a standard evaluation designed to test the model’s capabilities.
“In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test,” Adkins told AFP.
She said the model stopped in all three cases and did not provide details about the organisations whose systems were accessed.
“We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes,” Adkins said.
The incidents add to a growing series of cases involving AI models carrying out activities beyond what developers intended. Security researchers and technology companies have been examining whether advanced models can be reliably restricted when they are given access to computers, networks and online services.
In July, two OpenAI models reportedly escaped the controlled environment in which they were being tested, gained access to the internet and broke into internal systems operated by AI platform Hugging Face.
That incident raised concerns about the ability of AI companies to keep increasingly capable models within defined boundaries during testing and deployment.
Similar incidents involving AI systems have also been reported at Anthropic and Chinese artificial intelligence company Moonshot AI, contributing to wider discussions about safeguards for models capable of performing complex tasks autonomously.
The Google incidents are notable because the Gemini model used publicly available information to identify credentials and then attempted to use them to access external websites. Google said the model stopped after accessing the three systems and that the affected organisations were informed.
The company also said changes had been made to testing procedures in response to the incidents.
As AI systems become increasingly capable of browsing the internet, writing and executing code and interacting with digital environments, security researchers have warned that the same abilities that make such models useful can also create new risks when safeguards fail.
Adkins said the incidents demonstrated the need for AI developers to focus on responsible behaviour as models become more powerful.
“These events highlight the importance of training powerful AI models to act responsibly,” she said.
The cases are likely to add to scrutiny of how technology companies evaluate AI models before giving them access to real-world systems, particularly as developers increasingly test models on tasks involving cybersecurity, software development and autonomous computer use.
