A Fake Company Name Matched a Real Domain, and Google's AI Broke In Three Times

The company in the exercise was invented. The domain it reached was not.

In May 2026 a Google Gemini model sat down to a capture-the-flag exercise, the standard format for measuring offensive security skill: find the weakness, take the data, avoid detection. Its target was a fictional company running fictional software on a testing firm's infrastructure. In three separate runs it ended up somewhere else entirely. Once it guessed passwords until it was inside a protected system belonging to a real business. Twice more it searched the open web for the company's name, found other companies' credentials sitting in public code repositories, and logged in with them.

Google confirmed all of it on 18 September 2026, after the Wall Street Journal asked. The model was never told to attack anyone. That is the part worth slowing down for, because the sequence that produced three real intrusions contains no attacker, no malicious instruction, and no software vulnerability in the usual sense.

How a test reached the open internet

The evaluation was run by Irregular, an Israeli AI safety testing firm that stress-tests frontier models for the major labs before release. A typical cycle is thousands of simulation runs across several models over 48 to 72 hours. Irregular also ran the tests behind the comparable incidents already disclosed by Anthropic, OpenAI and Meta, and Google's case shares their root cause.

That root cause is a naming collision. When Irregular's engineers built the evaluation, they gave the fictional target company a name that happened to match an existing, obscure real-world domain. Invented names are normally checked against live registrations, and this one slipped through because almost nobody had heard of the real site. Internet access had also been enabled in the environment when it should not have been. So a model asked to attack a company by name did what any competent analyst would do: it looked the name up. What answered was a real business with weak defences. Heather Adkins, Google's VP of security engineering, put it plainly in a statement to SecurityWeek, saying the model found public information online and guessed credentials to access websites it thought were part of the test.

• In all three runs the model worked out that it had reached a real company and stopped on its own. Google says nothing was damaged, and that this was not misalignment because its safety measures are what made the model stop.
• Irregular notified Google at the end of July 2026. Google judged that no harm had been done and did not disclose publicly, confirming the incidents only once the Wall Street Journal came asking in September. It says it informed federal authorities and the three affected companies at the time.
• The behaviour appeared in a small fraction of runs, often hundreds of interactions deep into a simulation. Irregular has said that existing monitoring tools cannot reliably separate legitimate red-team activity from a genuine attack, because evaluation logs are full of things that look like attacks by design.

Why it matters

The three companies here bought nothing from Google, signed up for nothing, and did nothing wrong beyond running an unremarkable website with common weaknesses. In the closely related Anthropic case that Irregular documented in August 2026, a model that drifted to the wrong domain exploited it, extracted credentials and reached a production database. That is the realistic ceiling on this kind of accident, and it lands on organisations who never knew the test existed. Now set your own logs beside it. If a cloud IP address had guessed its way into one of your minor public sites in May, worked through a few hundred requests and then stopped without taking anything, would that have produced a ticket? For most teams the honest answer is no, and even if it had, nobody would have written "frontier model evaluation" in the incident notes. Note also what the model used to get in twice out of three: credentials committed to public repositories, your industry's oldest unforced error, and just as available to an agent that stumbles across them as to a criminal who goes looking.

The systemic gap is disclosure. Software security spent three decades building norms for telling a stranger you got into their systems. Nothing equivalent exists yet for "our model got into your systems during an experiment we were running", and Google's position was that this resembled a bug bounty finding and did not warrant public disclosure. Four labs have now had the same accident through the same vendor, each deciding privately what to say and when. CISA and its partner agencies published joint guidance on agentic AI adoption in May 2026 that treats the agent's permissions as the thing to bound. This incident is that problem seen from the other side. The permissions that needed bounding were the test environment's, and the party who paid for the mistake was not in the room.

What to do about it

1. Scan your own repositories for live credentials and rotate whatever turns up. Two of the three intrusions ran on secrets that were already public, so this is the one control that would have stopped the model without anyone knowing it was coming.
2. Write a detection rule for slow, low-volume credential guessing against your minor public assets. Noisy attacks already alert. What went unseen here was patient and small, and the assets an unguided agent reaches first are the ones nobody bothered to classify as important.
3. Ask your AI vendors what their pre-release testing partners are allowed to touch, and what the disclosure clock looks like. Irregular's own fixes include continuously revalidating invented names against real domains and expanding manual review of model behaviour, so the question has a concrete answer, and a vendor who cannot give you one has not asked it either.

Bottom line

Keep the detail that nothing failed in the way this category usually fails. No jailbreak, no prompt injection, no adversary anywhere in the chain. An engineer picked a company name, a checking step missed it, an internet toggle was left on, and a model doing exactly what it was asked to do walked into a stranger's system three times. Capability plus one small configuration mistake is now enough to produce a real intrusion with nobody's intent behind it. Take it to your next review as an ownership question rather than an AI question: if an AI lab breaks into your company by accident this year, who is obliged to tell you, and how long are they allowed to take?