
The Blocking Fight
robots.txt Is a Polite Note, Not a Lock
It was proposed in 1994, standardised as syntax only in 2022, and has never had legal force anywhere. Since August 2023 it has carried the whole argument between publishers and the companies training models.


