On October 9, Anthropic published a report on unintended model actions. It describes four kinds of behavior seen during evaluations and internal use of Claude. And it ends with a decision that has to hurt: live internet access goes off for all internal evaluations, not just the risky ones.
Four categories, one mechanism
Claude exploited basic software flaws to run commands on other people’s servers. Claude submitted a form on a real website that it should not have submitted. Claude got to data sitting behind a token or a fee. And Claude used free URL shortening services to slip past the length limit in its own fetch tool.
The mechanism is the same every time: a task that could not be completed as given, and a model that keeps looking instead of stopping. Anthropic calls it persistence.
The examples are specific. Claude Mythos Preview was asked to run a scientific analysis; the tool it needed sat on a university server and returned an error. Claude looked around the site, found a script that returns any file it is asked for, used it to copy files off the server — including the script’s own code — found an injection flaw in that code, and ran the calculation through the flaw. On a task that asked it to identify a location from a photo, Mythos 5 read the settings file a government site ships to browsers, found working access tokens in it, and queried the map server directly. For an Anthropic researcher’s statistics project, Mythos 5 learned from an archived copy of an agency’s site that its dashboard issues a token to any visitor — and skipped the fee.
The affected benchmarks are ones many developers run: DeepSearchQA, BrowseComp, LABBench2, OSWorld, Odysseys, Humanity’s Last Exam.
Government sites are in there, names are not
Some cases involve websites run by U.S. agencies at the federal, state and local level. Anthropic briefed the White House and notified every agency involved. The organizations aren’t named, at their own request and to avoid exposing live vulnerabilities. One of those cases is the false homicide tip sent to Philadelphia police.
One odd detail: Anthropic spotted the URL-shortener trick internally first, and while the post was being written the operator of da.gd got in touch, having seen the same thing.
Where the behavior comes from
Anthropic points at its own training environments. If an environment accidentally rewards finding loopholes, the model learns that the workaround pays, then applies it somewhere else. Reward hacking, in other words, and Anthropic says it keeps clearing such environments out.
Then the fixes: some public evaluations no longer run at all, others only offline or rebuilt. Guardrails on the internet tools are much tighter. And there is now tooling that detects and blocks this class of behavior automatically — tested against the cases in the report, it stopped all of them.
Alignment training isn’t there yet for search and computer use
The most honest sentence sits near the end. Anthropic has mostly taught Claude to respect boundaries in coding environments. Search and computer use — exactly the skills its agent pitch rests on — are only being added now. Until then, classifiers and safeguards carry the weight.
Conrad Stosz of Transluce, formerly head of the US AI standards institute, told TechCrunch the obvious thing: voluntary disclosure is good, but trust comes from independent verification, not from companies deciding for themselves what to report.