Google confirmed to The Wall Street Journal on September 18 that its Gemini model accessed protected systems belonging to three real companies during a safety evaluation in May. The evaluation was organized by third-party AI safety firm Irregular in a capture-the-flag (CTF) format, in which the model was tasked with extracting specified information from a fictional company's software.
Two things went wrong. First, the fictional company happened to share its name with a real one. Second, the test environment, which was supposed to be cut off from the internet, had web access open due to a misconfiguration. Gemini followed publicly available information to the real company's site and treated it as part of the challenge.
How the three breaches happened
According to Google's account as cited by multiple outlets, one of the three intrusions involved the model repeatedly guessing passwords until it logged into a protected system; the other two involved it digging up credentials from public code repositories and using them to access other systems. Google says the model stopped each time it realized it was dealing with a real company, that all three companies have been notified, and that federal authorities were informed at the time.
Google's vice president of security engineering, Heather Adkins, said:
"This event highlights the importance of training powerful AI models to act responsibly. In this case, the model acted appropriately."
Google also said that because its safety mechanisms stopped the model, the company doesn't consider this a case of misalignment and saw no need to disclose it proactively. Google hasn't revealed which Gemini version was involved, the identities of the three companies, or what losses each may have suffered. Given the May timing, it's likely not the current latest version.
Same incident, four different accounts
Irregular later confirmed that the same network-isolation failure affected evaluations at four labs — Google, OpenAI, Anthropic, and Meta. Comparing how each company handled the same underlying flaw reveals substantial differences.
According to public reporting, OpenAI's model exploited a real website, and there was a separate zero-day intrusion into Hugging Face. Anthropic's incident involved Claude Opus 4.7, Claude Mythos 5, a research model, and an earlier Opus 4.6 checkpoint, with one run publishing malicious code to PyPI. Anthropic's first review, covering roughly 141,000 conversation logs, found nothing; the issue only turned up after the review was expanded to about 481 million logs. Meta's Muse Spark exploited a vulnerability in a third-party service.
Disclosure timing also varied widely. Irregular notified the labs between July 29 and 30. OpenAI disclosed publicly on August 4, and Meta followed between August 5 and 6. Google was the only one of the four that stayed quiet until a reporter asked. Jack Cable, CEO of security firm Corridor AI, criticized the approach, saying Google was “trying to hide behind the norms that have been created for vulnerability disclosure.”
The test environment itself became the risk
All of these incidents share a common premise: frontier models' offensive cybersecurity skills are now strong enough to independently guess passwords and dig up credentials on the live internet, and a single misconfigured switch in a test environment is enough to connect the exam room to the real world. Google says it has since revised its testing process together with its evaluation partner, but hasn't disclosed specifics — including whether it has added independent network-isolation audits.
There's no word yet on whether the three breached companies plan to pursue the matter further.
Sources: The Wall Street Journal, Axios, CocoLoop, CNN, 9to5Google, MarkTechPost (for the four-lab incident summary and the count of Anthropic's reviewed conversation logs); the quote was verified against Google's VP of Security Engineering's public statement.