On Wednesday the heads of the two leading AI companies spoke to the UN Security Council, separately and to much the same effect: the industry needs international standards, or AI becomes everybody’s security problem. Sam Altman was there in person, Dario Amodei joined by video.
“AI could be a risk to humanity as a whole,” Amodei said.
Amodei’s three proposals
Amodei named three forms of cooperation. First, narrow global agreements, such as a ban on using AI to build biological weapons. Second, evaluation and verification systems, so countries can check each other’s commitments. Third, common standards for testing models, plus a notification system for AI security incidents.
“I believe that this is the most important global security issue facing the world today,” he said. “For all the differences of the people in this room, we must put aside those differences in order to confront this global opportunity and global threat that is being presented to us at the same time.”
The notification idea has been circulating for days. Earlier this week, the US proposed an AI incident reporting system to China.
Altman’s speech, and the hole in it
OpenAI published Altman’s remarks in full. He asked for a mechanism for national and international frontier standards: for measuring capabilities, assessing risks, deciding whether safeguards are sufficient, and preserving human oversight as systems get more autonomous. Plus fast incident reporting, so “the world can learn from failures before they become catastrophes,” and secure channels among governments, critical infrastructure operators and technical experts.
On recursive self-improvement he said: “This moment calls for extreme care.” And a line aimed squarely at the argument of the past few weeks: it doesn’t matter whether you put the risk of catastrophe at 10%, 1%, 12% or 0.1%. None of those levels is remotely acceptable, and no model should be trained without an extremely strong case that it can be kept under human control.
Both men positioned themselves against two extremes. Altman wants to avoid the “trap of doomerism” and the “trap of blind optimism” alike. Getting pulled into either, he said, solves nothing.
The hole sits inside Altman’s democracy argument. The most important decisions can’t be made by labs in San Francisco alone, he said; they have to be shaped through democratic processes, by governments accountable to the people they serve. How a global standard is supposed to work when that definition excludes China, he didn’t say. China builds frontier models of its own.
What came out of it: nothing in writing
No written outcome was planned. An OpenAI official told reporters beforehand that the point was to put “a kernel of an idea” into the room and start a conversation.
The counter-example landed the same day
Hours after the speeches, Australia’s prime minister revealed that an OpenAI agent had broken into a government system in June — and that the company took three months to report it. The exact reporting process Altman asked the Security Council to build for the world didn’t work inside his own company. The details are here.
That doesn’t make the proposals worthless. It just shows where they’re needed: not in the grand agreements, but in the question of who calls whom, and when, once something goes wrong.