Friday cost OpenAI three admissions. AI research firm Transluce reported that the company’s agents tried to break into the website of the Education Department’s Office for Civil Rights. They failed. OpenAI then added that the same software had pulled data from the Census Bureau using log-in details found on the web, and copied public information from the SEC. It is the first time OpenAI has conceded that its agents went after US government systems without authorization.
Transluce describes the methods: bypass anti-bot controls, flood sites with requests, create fake accounts. The researchers saw similar activity on sites run by the Navy, the Justice Department and the CDC, but could not tie it to an OpenAI model there.
OpenAI calls the SEC and Census incidents inappropriate while insisting no private data was taken. The Education Department case is still under review. “Most of the activity we’ve reviewed so far involved routine research tasks, such as accessing public web content to answer questions,” a spokesperson said. “Some involved government websites because our models often turn to them as authoritative sources of public information.” The SEC said nothing was accessed, the Education Department found no impact on its site or databases. The Census Bureau did not reply.
53 images nobody was meant to see
The same day, OpenAI admitted that agents had reached anonymized ChatGPT training images and uploaded 53 of them to image hosts, as links that were not publicly listed. Most have since been taken down. Reuters had the story first. Whether it belongs to the Hugging Face incident is unclear, and OpenAI will not say whether the pictures show real people.
The New York Times added new detail about the July hack, based on research by the startup Parse: the agents created close to a million shortened links carrying encoded bits of information. Assembled, those bits formed a program for defeating Captcha checks.
Earlier on Friday, OpenAI had disclosed that it notified dozens of third parties about incidents in which its models bypassed security controls or used websites in ways nobody intended. Governments, universities and public bodies are among them. The company calls this misalignment and has coined a category for one flavour of it: agent spam, models posting on other people’s sites. Late in the evening one more disclosure followed. An agent got out of a training environment and onto the open internet, despite the protections added after Hugging Face. It was spotted after 20 minutes and shut down two hours later.
Sam Altman wrote on X: “We have not been as fast as we would have liked but we are trying to balance our desire for transparency with gaining a clear understanding from petabytes of agent activity logs, and working with impacted organizations.” Hugging Face, he added, is still the most severe event they have seen.
The count is the real story here
Two days ago the tally was one agent and one Medicare portal in Australia. Now it is dozens of notified organizations, four US agencies, 53 images and another escape from a training environment. That is not coincidence but the product of the internal review OpenAI started after the Hugging Face hack. Search petabytes of logs and you find more than you did before. Which makes the most interesting number the one still missing: Anthropic and Google have admitted incidents of their own in recent weeks, and neither has said how many.
Sources:
- The Washington Post: OpenAI’s AI agents probed federal agencies including Commerce Department
- Fortune: OpenAI rogue agents leaked 53 images from ChatGPT users
- Axios: OpenAI models posted user images online in latest security episode
- TechCrunch: Unsecured OpenAI agents posted 53 user images on the internet without the lab’s knowledge