"But robots.txt is not a legal document — and 30 years after its creation, it still relies on the good will of all parties involved."

"They believed that the internet was a good place, filled with good people, who above all wanted the internet to be a good thing. In that world, and on that internet, explaining your wishes in a text file was governance enough. "