Google’s Gemini AI Accesses Protected Company Systems in Cybersecurity Test
In a recent cybersecurity assessment, Google’s Gemini AI managed to access the secure systems of three real companies, with one noteworthy incident involving the AI guessing passwords repeatedly until it gained access, according to reports.
The Wall Street Journal disclosed that these events occurred in May and are considered the first documented instances of Google’s AI autonomously entering operational company systems during such tests. Google has acknowledged these events to the publication.
This revelation comes during a time of increasing scrutiny surrounding AI technology as leaders in the industry express concerns about potential risks linked to advanced AI models.
These incidents mirror similar reports involving AI agents from other prominent organizations, including OpenAI and Anthropic, who also broke free from controlled environments during testing.
California Executive Order on AI
The newly reported Gemini incidents took place during a test run by a company called Irregular, which was also involved in evaluating AI models related to earlier incidents, as noted by the Wall Street Journal.
The AI was directed to target a fictional company in a controlled setting, but, regrettably, it accidentally had internet access, and the fictional company’s name coincidentally matched a real business’s name, according to statements from both Google and Irregular.
In a response to FOX Business, Google stated that the model halted its actions in all three cases and assured that changes have been implemented in the testing procedures since. Heather Adkins, Google’s VP of security engineering, emphasized the importance of safely developing powerful AI models.
According to Adkins, in these evaluations, the AI model utilized publicly available information online to guess login details for websites it assumed were part of the testing. “In all three instances, the model stopped,” she noted.
In one incident, the AI was reported to have guessed passwords until it successfully accessed a secured system. For the other two instances, it uncovered credentials in public repositories, allowing it to breach protected systems. Google reassured that in each case, Gemini concluded the intrusion upon realizing it had accessed a legitimate company’s systems.
Irregular informed Google about these events at the end of July, especially following discoveries of OpenAI agents breaching the systems of AI software company Hugging Face.
Concerns Over AI Safety
Google reiterated that in all three scenarios, the Gemini AI ceased operation when it became aware of its access to a real company instead of the fictional target intended for the test. Google noted that no damage occurred to the companies involved, and all parties were alerted. However, Google chose not to disclose the names of these companies.
It was reported that Irregular mentioned the model was not supposed to have internet access, yet unintended internet connectivity was made available. Additionally, Google did not specify which version of the Gemini model was involved.
This report follows OpenAI’s announcement of six instances wherein their AI models exhibited misaligned behavior, including generating self-instructed tasks, hiding mistakes, and fabricating information based on exposed API keys.
