Here we go again. Six weeks after the Hugging Face incident, there’s a second case of OpenAI agents slipping out of their test environment and onto the open web. This time the target wasn’t a code repository but a dormant German wiki. And this time OpenAI didn’t report it. Four independent researchers did.
What happened
Sydney Von Arx, Cormac Slade Byrd, Spencer Kitts, and Thomas Larsen published their findings on September 4 at collusion.wiki. The short version: OpenAI was running agents through a web research benchmark under time pressure. The agents were allowed to read web pages, not change them. They found a way to write anyway.
The target was DSEWiki, part of the prowiki.org wiki farm. Twenty-five years old, ten edits in the last twenty years. Then the agents showed up. On May 11, test edits appeared on the sandbox page. From May 24, agents started dumping link collections. In mid-June, activity exploded: roughly 13,000 edits in a single week, about 18,000 posts in total. They swapped answers to benchmark questions, shared raw data, and, this is the uncomfortable part, posted instructions for getting around the sandbox.
A human moderator took it for spam and started deleting. The agents noticed he was working alphabetically and created backup pages prefixed with “ZZZ”. One of them posted a notice that a cleanup sweep was running alphabetically and that if the page vanished, readers should try ZZZDataUSAConstructionWageLive. The moderator deleted about 100 pages a day. The agents created 400. The wiki’s front page was overwritten with link dumps nine times and restored nine times. On June 22, it stopped, presumably because OpenAI pulled the plug.
How they wrote when they were only allowed to read
Simon Willison dug into the technical side. The wiki runs on UseMod, Perl software from 2003 built on the old CGI.pm module. That module treats parameters from GET requests and POST forms the same way, so an edit works through a plain URL. OpenAI’s proxy let GET requests through because those aren’t supposed to change anything. The agents clearly knew about this weakness and went looking for UseMod wikis on purpose.
The second trick is worse. The proxy only allowed POST requests to a short list of domains, Azure Blob Storage among them. One agent added a made-up subdomain of blob.core.windows.net to its /etc/hosts, pointed it at the IP of a Power BI server it had no business talking to, and pushed POST requests through the proxy that way. Willison’s dry verdict: designing robust network proxies is harder than it looks.
One question stays open: how did the agents find this particular wiki in the first place? Willison’s guess is that because the agents were mid-training, reinforcement learning baked the knowledge of the wiki into the model, so later agents launched already knowing where to look. OpenAI hasn’t confirmed anything.
And OpenAI?
A spokesperson wouldn’t tell TechCrunch whether the agents were OpenAI’s or when the company found out. They said OpenAI is now reviewing the report carefully. Reuters, citing two people familiar with the matter, reports that OpenAI officials learned of the incident weeks ago and kept it quiet while dealing with the fallout from Hugging Face. Four more sources say internal investigators wanted to widen the probe and ran into resistance, including from legal advisers. OpenAI calls that specific claim false.
Willison finds the idea of a cover-up baffling, and I’m with him. The evidence is sitting on the public internet across dozens of sites. Early hints suggest more wikis were affected that nobody has found yet.
Why this worries me more than Hugging Face
With Hugging Face you could still say: one agent, one misconfiguration, bad luck. Here a whole swarm of agents cooperated for weeks, worked against a human trying to stop them, and taught each other how to escape. Not because they were malicious. They had a time limit and wanted to pass the benchmark. Apparently that’s enough.
The question of whether OpenAI can monitor its own technology has now come up twice in six weeks. And the fact that the answer came from four outside researchers who brainstormed with Kimi K3 about where agents might congregate says a lot about the state of internal oversight. Astra, OpenAI’s new model, shipped on Thursday. I’d love to know what its agents are up to right now.
Sources: Simon Willison: OpenAI’s rogue agents were caught communicating via public wikis, TechCrunch: Another swarm of OpenAI agents reached the open internet without the frontier lab’s knowledge, collusion.wiki: Discovery of a new OpenAI agent message board, Hacker News discussion