Hacker News
- Go ahead and block AI web crawlers https://coryd.dev/posts/2024/go-ahead-and-block-ai-web-crawlers/ 22 comments
Linking pages
- Access to data isn't a grant to exploit it // Cory Dransfeldt https://coryd.dev/posts/2024/access-to-data-isnt-a-grant-to-exploit-it/ 6 comments
- Shields up https://coryd.dev/posts/2024/shields-up/ 1 comment
- Robots.txt Won't Save You—Ryan Bagley https://rb.ax/blog/robots.txt-wont-save-you/ 1 comment
- The Omg Lol Times · Blocking Bad Bots https://omglol.news/2024/06/16/blocking-bad-bots 0 comments
Linked pages
- https://www.reuters.com/technology/reddit-ai-content-licensing-deal-with-google-sources-say-2024-02-22/ 77 comments
- Dark Visitors – A List of Known AI Agents on the Internet https://darkvisitors.com 67 comments
- The New York Times blocks OpenAI’s web crawler - The Verge https://www.theverge.com/2023/8/21/23840705/new-york-times-openai-web-crawler-ai-gpt 19 comments
- Now you can block OpenAI’s web crawler - The Verge https://www.theverge.com/2023/8/7/23823046/openai-data-scrape-block-ai 3 comments
Related searches:
Search whole site: site:coryd.dev
Search title: Go ahead and block AI web crawlers • Cory Dransfeldt
See how to search.