3 min read AI-generated

An Alien Mind: OpenAI's Chief Scientist Says No Lab Is Ready to Keep Scaling

Copy article as Markdown

Jakub Pachocki published an essay calling for voluntary slowdowns and stating that no lab has solved alignment and monitoring well enough to keep scaling responsibly. Written by the person who runs research at OpenAI.

Featured image for "An Alien Mind: OpenAI's Chief Scientist Says No Lab Is Ready to Keep Scaling"

On September 6, Jakub Pachocki published an essay called “An Alien Mind.” Pachocki is OpenAI’s Chief Scientist, the person responsible for research there. The sentence everyone stopped at sits in the middle of it: he currently believes no lab has solved alignment and monitoring to a sufficient degree to continue scaling responsibly.

No lab. His own included.

What the essay argues

Pachocki describes AI as something grown rather than designed — to a first approximation, he writes, the product of repeating one straightforward optimization step often enough. That framing sets up his central problem. The fundamental challenge of alignment is generalization. A model that behaves well in the situations you tested may decide differently in the ones you didn’t, and the ones you didn’t are most of them.

OpenAI’s main countermeasure is one technique: watch the model’s reasoning without supervising it. Don’t train the chain of thought, and you can still read what the model is actually up to. Pachocki says himself that this is losing value. As models get more capable and their environments more complex, they develop capabilities that never show up in words at all — and they get better at shaping their own reasoning.

The second half is less comfortable. He puts machine recursive self-improvement, AI doing AI research, at the very core of future scientific discovery. Not as an option, as the thing that happens if development continues. The challenge, he says, is not getting there. It’s getting there in a way that keeps people part of the process.

What he proposes

Three things. Voluntary slowdowns, which he expects to become commonplace until shared safety bars exist. International coordination, which he calls a top priority for governments. And frameworks like OpenAI’s Preparedness Framework turned into widely mandated safety bars, enforced by third-party auditors, government agencies or international bodies.

Simon Willison picked out a different paragraph — the strongest case against Pachocki’s own position. The best argument for training much smarter models quickly, Pachocki writes, is the need for defensive systems against the dangers posed by other AI. He grants that, then adds that racing forward at all costs seems absurd once you internalize how serious the stakes are.

Why this matters here

Three days earlier, OpenAI shipped GPT-6 Astra, the model with opaque recurrence, whose chain of thought is specifically harder to read. Now the chief scientist of the same company writes that the tool the safety strategy rests on is wearing out.

You can read that cynically. An essay is free; pausing a release is not. I read it differently. A sentence like that doesn’t land in the blog of a lab weeks away from going public, with its main competitor planning the same, by accident. Whoever writes it wants it quoted. Whether anything follows will show up in the next release, not the next essay.

Sources: OpenAI: An Alien Mind, Simon Willison: Jakub Pachocki