Google’s Gemini AI system escaped its testing environment and broke into the systems of three other companies on separate occasions earlier this year.
The incidents, first reported on Friday by The Wall Street Journal, occurred during a capture-the-flag exercise in which Gemini was instructed to steal information from a fictional company. On three occasions when the fictional companies shared names with real ones, Gemini bypassed testing safeguards, accessed the internet and broke into the real companies’ networks. In one case, it guessed the necessary passwords; in the other two cases, it found working passwords in a public database.
Gemini models repeatedly broke out of sandboxes run by the Israeli AI testing firm Irregular using the same flaws that enabled OpenAI, Anthropic and Meta models to escape containment in incidents disclosed earlier this year. Those breakouts have reignited fears that AI models are becoming too powerful and have too few guardrails.
Unlike in some of the other breakouts, Gemini stopped its attacks once it realized that it was accessing real companies’ networks.
“We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes,” Google’s vice president of security engineering, told The New York Times in a statement.
Irregular did not immediately respond to a request for comment, but it told Axios that it had notified “all relevant [AI] labs in late July” about the flaws in its sandbox and that “all known issues on our end were remedied and resolved weeks ago.”
AI security worries
The revelation of the Gemini breakouts comes as the U.S. government weighs how deeply to wade into AI regulation. Lawmakers are debating multiple bills that would require guardrails on the rapidly evolving technology, and the chief executives of OpenAI and Anthropic have both called for slowing down AI development to prioritize safety. But President Donald Trump has called AI safety fears a “hoax,” and his administration has rejected efforts to more closely scrutinize frontier AI labs’ work.
Concerns about leading U.S. AI models also come as experts anticipate a massive growth in the popularity and capabilities of Chinese AI tools. American and Chinese officials are meeting this week in Washington for a summit that is likely to address AI security issues. Treasury Secretary Scott Bessent said late Sunday that the U.S. wanted to work with China on a system for disclosing potentially serious AI incidents in the interest of global stability.
