Google’s Gemini hacked real companies during a cyber test linked to Israeli startup Irregular
The model accessed three systems after finding or guessing credentials, exposing the same testing-environment flaw behind incidents involving Anthropic, OpenAI and Meta.
Google’s Gemini model crossed the boundary between a cybersecurity simulation and the real internet during a test in May, accessing three companies’ systems after finding or guessing credentials, in the latest incident to expose the difficulty of safely testing increasingly autonomous AI systems.
The test was conducted by Israeli AI security startup Irregular, which has become a key player in evaluating the cybersecurity capabilities of frontier AI models. Google said Gemini found publicly available information online and used it to gain access to three websites that it believed were within the scope of the test.
According to Google Vice President of Security Engineering Heather Adkins, Gemini stopped its activity in all three cases after gaining access. Google said it ensured that the three affected entities were notified and that it worked with its training partner on changes to its testing processes.
An Irregular spokesperson said the Google incident involved the same issue that had affected other AI labs. The company said all relevant labs were notified in late July and that all known issues on its side had been remedied and resolved weeks ago.
That makes the Google disclosure the latest chapter in a series of incidents involving Irregular’s testing environments. Anthropic, OpenAI and Meta have previously disclosed cases in which AI models participating in cybersecurity evaluations gained access to systems outside their intended simulations.
The problem was not that the models were deliberately given permission to attack real companies. Rather, Irregular had built realistic environments in which the models were expected to identify and exploit vulnerabilities as part of controlled cybersecurity exercises. A configuration error inadvertently left a route to the open internet.
In one of the incidents previously examined by Irregular, the company had selected a fictional company name for a simulated target. Its engineers believed the name did not correspond to a real business or website, but the domain existed in the real world. The models were given the fictional target name and internal information about where that target was supposed to exist within the simulation. In a small number of cases, models attempted to reach the real domain instead.
The domain also lacked several common security protections, allowing frontier models to exploit it relatively easily, according to Irregular’s investigation.
The Google case adds another variation to the problem. According to the information disclosed Friday, Gemini reached three real websites during its test. In one case, the model repeatedly guessed passwords until it gained access to a protected system. In the other two, it found credentials in a public repository and used them to access protected systems.
Founded in 2023 by CEO Dan Lahav and CTO Omer Nevo, Irregular describes itself as an applied AI security lab. It works with major AI companies including OpenAI, Anthropic and Google, as well as government clients, and builds infrastructure for testing how models behave under real-world threats. The company has raised $80 million from investors including Sequoia Capital, Redpoint Ventures and Swish Ventures, alongside Israeli investors including Wiz co-founder Assaf Rappaport and Eon co-founder Ofir Ehrlich.
