Enterprise & Security

New York City Council: Anthropic, OpenAI and Google only came after a subpoena threat, SpaceXAI not at all

3 min read AI-generated

Speaker Julie Menin asked about liability when a model causes serious financial harm or a death. Then she noted that none of the four companies can put a number on cataclysmic risk.

Featured image for "New York City Council: Anthropic, OpenAI and Google only came after a subpoena threat, SpaceXAI not at all"

On Monday, October 5, a New York City Council committee held a hearing on the risks and regulation of AI — under oath, by video link, in a packed room, with protests in City Hall Park outside. A city. Not a state, not Congress. That’s the part worth noticing.

Anthropic sent Logan Graham, head of its Frontier Red Team. OpenAI was represented by Morgan Dwyer, who runs policy development and operations; Google by Alice Friend from its AI policy side; Meta by Shane Cahill, its policy director for legislation. Meta confirmed it would attend in late September. Anthropic, OpenAI and Google, by Menin’s account, only agreed once the council threatened to subpoena them.

Elon Musk’s SpaceXAI didn’t show, despite a subpoena the city issued last week. Menin calls that a direct violation and plans to take it to court.

What the people who left had to say

Jacob Coxon, formerly at OpenAI and at Anthropic until September, testified voluntarily. I wrote about him in September, when he quit saying the labs were gambling with our lives. In front of the council he didn’t soften: “On the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction.” He called the industry’s approach “extremely reckless”. Move fast and break things works for a photo-sharing app, he said, not for building the most powerful technology ever built. To Bloomberg he put it shorter: we do not know how to control any AI system yet.

Daniel Kokotajlo, formerly of OpenAI, testified under subpoena. His point isn’t doom, it’s measurement: “Our ability to even notice misalignment problems is already quite poor and is set to get much worse in the near future.” The field, he said, is more like psychology than engineering. Safety patches he described as duct tape that falls off later.

Alex Turner, formerly of Google DeepMind, turned the usual China argument inside out. Asked by Menin how to balance frontier development against the race with China, he answered: “China is not our only potential adversary. With reasonably high chance, we are racing to build and grow our own adversary here at home, which is misaligned AI.”

The answers that didn’t arrive

Menin lost patience more than once over the lack of direct answers, including on liability for serious financial harm or deaths. Her summary: she would take it, then, that none of the four companies can quantify the risk of something cataclysmic — answers she called “troubling at best.”

Google’s Friend argued for a comprehensive federal framework, citing national security. Her line: if it’s illegal without AI, it’s still illegal with AI.

Menin opened with a swipe at Washington, after the voluntary accord Trump signed with tech leaders last week: the idea that AI is going to self-regulate, she said, defies all reason.

A city has subpoenas, Congress has pledges

What sets this hearing apart from the dozens before it isn’t the content. Coxon’s and Kokotajlo’s argument has been in the open for weeks. The new thing is the instrument: a city council pulled three of the four labs to the table with a threat, and is going to court over the fourth. At the federal level, last week produced a voluntary accord.

Whether New York turns any of this into law is open, and a city’s authority over frontier models is thin. But if regulation starts where someone can actually compel attendance, October 5 was a start.

Sources

AnthropicOpenAIRegulationAI SafetyNew York