If you do not want your website feeding model training today, you have to know every crawler by name and keep separate rules for every vendor in your robots.txt. A shared signal is meant to replace all of that: one entry, one statement, every participating provider.
It was proposed by Kevin Kelley, technical lead for standards at Anthropic, inside the Internet Engineering Task Force’s AI Preferences working group. OpenAI, Google, Microsoft and Anthropic said jointly on October 30 that their crawlers will respect the signal and stop reading a site for training when they encounter it.
What OpenAI committed to, and what it did not
OpenAI’s own post is careful on two counts. Implementation is coming “in the months ahead”, with details in a later update. And the new signal does not replace the crawler controls OpenAI already offers; it sits beside them. The whole arrangement is voluntary anyway: an IETF draft binds nobody, and a robots.txt is not technically a lock, it is a request.
Meta communications head Andy Stone signaled support without saying whether Meta will honor the signal. That is the distinction that counts.
The ground is already occupied
This proposal does not land in an empty field. Really Simple Licensing lets publishers charge crawlers that visit. Cloudflare has been blocking AI crawlers by default for a while. Both attack the same point and want more: not merely to express a preference, but to enforce it or get paid for it.
A shared signal solves exactly one problem, and it is an administrative one — the work of telling a dozen companies the same thing separately. For publishers who want to sell their content, that is nowhere near enough. For whoever runs a small site and just wants to be left alone, it is the only thing that realistically works.
The proposal coming from Anthropic is the interesting part
Four companies currently fighting in court over training data have agreed on a signal that takes data away from them. There is a reading for that: a voluntary standard the industry writes itself is cheaper than one a legislature writes. That Anthropic held the pen fits a company that likes to present its position on copyright as a principle. Whether it is more than a principle gets decided on the day the signal actually ships inside the crawlers — and OpenAI has deliberately set no date for that.