"The Crawler Zoo" - a live zoo of the bots & crawlers visiting a test-site, with a trap for the ones that ignore robots.txt
4 days ago by JohnnyEnzyme to c/technology
I found this one just too fascinating not to share, even though it originally appeared at the evil empire via a post by u/Time_Instruction_955.
A live zoo of the bots visiting this site: 7,611 so far today. See what GPTBot, ClaudeBot and Googlebot are, and how to block them.
As a nature-buff, I'm also a bit fascinated by the way it organizes things in a Linnean classification structure (i.e. family, genus, species, etc). Super-clever IMO.
See what you think!
Now this is a good technology post! I can't wait to implement this on my blog to keep the pesky bots away...I hate how nasty the internet has become, glad there are still ways to fight back.
@piefed.social
Tech related news and discussion. Link to anything, it doesn't need to be a news article.
Let's keep the politics and business side of things to a minimum.
Rules
No memes
go to feed...
@piefed.social
Tech related news and discussion. Link to anything, it doesn't need to be a news article.
Let's keep the politics and business side of things to a minimum.
Rules
No memes
go to feed...
I really love this, it’s so well executed.
save