Anthropic & Claude

METR at Anthropic: Six Evaluators With Company Hardware, Desks and Employee-Like Access

2 min read AI-generated

Anthropic is building this as a three-tier ladder. SecureBio is reviewing the chemical and biological weapons chapter of the August Risk Report, and talks with Hugging Face are still early.

Featured image for "METR at Anthropic: Six Evaluators With Company Hardware, Desks and Employee-Like Access"

Anthropic said on Wednesday that a team from the nonprofit METR now has “employee-like access”: internal tools and systems roughly on par with what Anthropic’s own risk assessment teams use. Six METR people are set up with company hardware and office space. METR publishes the results of its risk assessments itself.

What “like an employee” means

Anthropic staff are encouraged to talk to those six the way they would to a colleague working on alignment risk. A few clearly defined categories are carved out: customer data, attorney-client privileged information, and details of model architectures.

The interesting part is publication. METR analyzes risk levels, incidents and practices, and publishes. Anthropic holds no veto. It can require redactions, but only within the predefined categories, and METR may describe publicly that something was redacted and whether it considers the redaction justified, as long as it doesn’t reveal the content.

The siloing runs the other way too. Only this small team works under the agreements; other METR employees are walled off, including researchers collaborating with Anthropic on separate projects.

Three tiers, and METR sits on the third

Anthropic is building this as a ladder. On tier one, evaluators run narrowly scoped investigations, Anthropic decides what access to grant, and publication needs approval. On tier two, approval drops away, access widens, and redactions in fixed categories remain possible. Tier three is the METR arrangement.

More partners are in the pipeline. SecureBio is reviewing the chemical and biological weapons chapter of the August Risk Report, after completing and publishing a pilot review of the previous one. With Accenture and Faculty, the work covers red-teaming, alignment assessments and safeguard testing. Talks with Hugging Face are early.

The banking analogy is Anthropic’s own

The post draws the parallel to bank supervision: because a bank knows its own risk positions better than any outsider, regulators station resident examiners inside the largest institutions. That’s a notably frank comparison, because it describes oversight rather than voluntary transparency.

Two things stand out to me. First, the no-veto clause is the sentence the whole arrangement should be judged on; everything else is access logistics. Second, this is a commitment, not a law. Anthropic pledged under the White House Accord on Super Intelligence to work with independent evaluators, and is now showing what that looks like in its own house. What’s missing is METR’s first published report, which will show how much of that access survives contact with practice.

Sources

AnthropicMETRAI SafetyAuditingTransparency