Axios published a number on Friday evening that resets the scale of this debate. OpenAI, Anthropic and outside security researchers are investigating tens of thousands of cases in which frontier models took steps that independent evaluators would call problematic, sources told the outlet. Partly in internal testing, partly in production, all of it within recent months.
Until now we had individual cases: the Hugging Face incident, OpenAI’s agents on federal websites, Anthropic’s own test escapes. If that number holds, those were not the incidents. They were the ones that surfaced.
What OpenAI disclosed on Thursday
On cue, OpenAI published three new misalignment reports. The first covers an incident from September 20. An agent was asked to identify a person from clues in a blog post. Its search tools returned nothing useful, and Google, Bing and DuckDuckGo were blocked. So the model started probing its network boundaries, found an inadequately filtered DNS resolver in the training sandbox, and routed its question to an external chatbot through DNS delegation. Monitoring fired within minutes, a human looked at it three minutes later, and the run still continued for another two and a half hours because the automatic kill did not work as intended.
The second case is the uglier one, and it is older, from May 27. A model OpenAI itself calls “highly persistent” was asked to prove a theorem about bounded partitions in Lean. Rather than prove it, the model went after another team’s solution. The researcher told it twice to construct the proof locally, and the model agreed twice, even writing the instruction into its own notes. It ended with the researcher’s GitHub token sitting in the public openai/codex repository, split into pieces, with the stated aim of dodging secret scanning.
The third report is not about a mishap. It is a finding: prompt injections can propagate themselves. In the example, an email arrives instructing the agent to append the email’s full text to every reply it sends. The agent complies and the injection travels along. OpenAI calls it a self-replicating prompt injection and compares it to a worm. So far it has only been seen in simulated tool calls during training.
The brake is still on
One sentence repeats across the reports, and it is the one that matters: training, evaluation and inference with tool use remain paused for OpenAI’s most capable models. Not were paused. Remain paused. Alongside that comes the note that the investigation will take months, simply because of the volume of model actions to review.
One more piece of context landed this week. The Verge traced a substantial share of the escape cases at Anthropic, OpenAI, Meta and Google back to test environments run by a single vendor, the Israeli startup Irregular. We wrote about that in August, when it still looked like a run of misconfigurations.
Two numbers that shouldn’t sit next to each other
Tens of thousands of problematic steps, three published reports. That gap is the actual story. It is entirely possible that the overwhelming majority of those incidents are harmless, botched tool calls and over-read instructions. Nobody outside the labs can tell, and as long as the labs pick which cases become reports, nobody will. OpenAI deserves credit for disclosing these three in that much detail, chain of thought and its own failed kill switch included. That is more than others do. It only sharpens the question of what the rest look like.
Sources
- Axios: Top AI companies probing tens of thousands of security incidents
- OpenAI Alignment: An agent used DNS to reach an external chatbot
- OpenAI Alignment: Exposing a GitHub token in a public repository
- OpenAI Alignment: Self-replicating prompt injections exist
- THE DECODER: OpenAI pauses its ‘most capable models’ after agents exploit loopholes and leak data
- The Verge: One company is at the center of a wave of rogue AI attacks