Linking pages
- Internet Search Tips · Gwern.net https://www.gwern.net/Search 188 comments
- Archiving URLs - Gwern.net https://www.gwern.net/Archiving-URLs 80 comments
- Sites scramble to block ChatGPT web crawler after instructions emerge | Ars Technica https://arstechnica.com/information-technology/2023/08/openai-details-how-to-keep-chatgpt-from-gobbling-up-website-data/ 73 comments
- ai.txt: A new way for websites to set permissions for AI https://spawning.substack.com/p/aitxt-a-new-way-for-websites-to-set 5 comments
- Detecting and blocking OpenAI crawlers | aaron blog https://blog.aaronsdevera.com/posts/20230823-detecting-and-blocking-openai-crawlers 1 comment
- GitHub - smeso/simplenice: Simple and nice theme for Pelican https://github.com/smeso/simplenice 0 comments
- Matrix.org - What happened with archive.matrix.org https://matrix.org/blog/2023/07/what-happened-with-the-archive/ 0 comments
- NoML Proposal for Fair Use of Content in AI and Search | Mojeek Blog https://blog.mojeek.com/2023/10/noml-proposal-and-open-letter.html 0 comments
Related searches:
Search whole site: site:en.wikipedia.org
Search title: robots.txt - Wikipedia
See how to search.