Enterprise & Security

Anthropic S-1/A: Recursive Self-Improvement Is Now a Risk Factor

2 min read AI-generated

In the same passage, Anthropic says it is considering updates to its Responsible Scaling Policy to keep humans in control of frontier model development.

Featured image for "Anthropic S-1/A: Recursive Self-Improvement Is Now a Risk Factor"

While everyone looked at the new quarterly figures, Anthropic slipped something else into Monday’s amended prospectus: two new sentences in the Risk Factors section saying the company may “imminently” meet its own definition of dramatically accelerated progress driven by AI. That is recursive self-improvement, or RSI.

What is actually new in the filing

Two statements, both added on November 2. The first is about the past:

In addition, our increasing use of AI systems to accelerate our own research and development may cause the capabilities of our models to advance more rapidly than our ability to develop, validate, and implement corresponding safety, alignment, and security measures. As a result, we have in the past not identified, and may in the future not identify, a significant acceleration in our rate of development until after it has occurred.

The second is about what comes next: internal estimates and expectations about future models—explicitly not current ones—suggest Anthropic may soon meet its Responsible Scaling Policy’s definition of “dramatically accelerated progress attributable to AI.” No current model has been designated as crossing that threshold, and the company says it is still assessing the underlying data.

A Responsible Scaling Policy change is on the table

The same passage says Anthropic does not anticipate being able to safely use Claude to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding, even if that becomes technically possible. So it is considering updates to the Responsible Scaling Policy “to keep humans in control of frontier model development at Anthropic.”

That is unusually specific for a document whose main job is limiting liability. Jack Clark put RSI in 2027 back in October. The prospectus now says “imminently.”

Two readings, and both sit in the same paragraph

An S-1 is disclosure and sales pitch at once. Read it skeptically and the wording is marketing: a company on the verge of building a model that builds models is a good story for investors. Read the liability side and you get the opposite. A firm writing that it failed to recognize capability jumps in the past is not doing that for promotional reasons in an SEC filing.

The second reading wins me over, and the reason is the tense. The sentence about the future is careful and stuffed with hedges. The sentence about the past is a plain claim of fact: “we have in the past not identified.” That is the half nobody is quoting, and it is the half that cannot be walked back.

Sources

AnthropicSafetyRSIIPO