For two weeks now the big red button has been at the centre of the debate. Congress wants one, California is studying one, and the public treats it as the obvious answer. CNBC asked security researchers and infrastructure people what such a switch would have to look like in practice. The answers are sobering.
What is actually on the table
A Kill Switch Act in the House, introduced this summer after the Hugging Face breach: it would give the Department of Homeland Security emergency authority to force labs to throttle or shut down models. A similar move in the Senate was killed off this week. And on Friday Governor Gavin Newsom signed an executive order creating an expert group to write a safety guide for California — a kill switch is one of the items they are meant to examine.
The redundancy problem
Mark Nitzberg runs the Center for Human-Compatible AI at Berkeley. His objection is unglamorous, which is exactly what makes it awkward: data centres are built to survive failure. “Our kill switch has to turn off the main systems and the redundant systems as well.” And if that works, the same infrastructure carries things nobody wants switched off. Nitzberg names the power grid and the financial system, both of which would be left exposed.
Tim Brown, formerly security chief at SolarWinds and now at venture firm Team8, puts it shorter: “There’s not one entity to kill. There are thousands of entities to kill.”
Pull too hard and the business stops
Ed Jennings, chief executive of security firm Darktrace, moves the problem from engineering into daily operations. “You have to be very surgical in that kill switch, in the remediation itself, because if you’re too broad or too extensive, well, then you shut down the business.” A switch nobody dares press is not a switch.
Then there is pace. Raj Rajamani of JetStream Security points out that by the time a law has been drafted, the technology has moved on.
The objection to the framing itself
Dylan Baker of the Distributed AI Research Institute thinks the whole framing is wrong. The term is vague on purpose, he argues, and companies can bend that vagueness their way. His proposal: safeguards modelled on data privacy, child safety or tobacco regulation, rather than a button nobody ends up pressing.
The idea is not written off entirely, though. Brown believes kill switches can work if they are built in from the start and stop protocols are standardised across labs. Nitzberg’s line: “I would say with some hope that it’s not too late.”
The button is the wrong metaphor
A kill switch comes from the factory floor. There is a machine, it has a circuit, and the button breaks it. A model has none of that. It runs on hardware serving a thousand other purposes, in data centres on several continents that cover for each other, and its weights are a file you can copy.
What stands out is who is saying this. These are not accelerationists fending off regulation — they are people who build and sell security systems. They are not saying a kill switch is pointless. They are saying the version currently being voted on is not one. Nick Warner of Neo gets it into a sentence: “It’s not too little, but it’s probably too late.”