Leebra opensource
login register

The Open-Source Software Saving the Internet From AI Bot Scrapers

a year ago by fattyfoods to c/opensource

save

share

report

Anubis, which block AI scrapers from scraping websites to death, has been downloaded almost 200,000 times.
load all comments
unexposedhazard 128 points a year ago

Non paywalled link https://archive.is/VcoE1

It basically boils down to making the browser do some cpu heavy calculations before allowing access. This is no problem for a single user, but for a bot farm this would increase the amount of compute power they need 100x or more.

save

path: 0 18113377, hotness: undefined, score: 128, children: 10
Mubelotix 78 points a year ago

Exactly. It's called proof-of-work and was originally invented to reduce spam emails but was later used by Bitcoin to control its growth speed

save

path: 0 18113377 18113540, hotness: undefined, score: 78, children: 7
RandomTester 4 points a year ago

it wasn't made for bitcoin originally? didn't know that!

save

path: 0 18113377 18113540 18122011, hotness: undefined, score: 4, children: 3
0xebfe 11 points a year ago

save

path: 0 18113377 18113540 18122011 18123307, hotness: undefined, score: 11, children: 2
RandomTester 12 points a year ago

you know it's old when it doesn't have ssl

save

path: 0 18113377 18113540 18122011 18123307 18123546, hotness: undefined, score: 12, children: 0
daw 3 points a year ago

TIL

save

path: 0 18113377 18113540 18122011 18123307 18126676, hotness: undefined, score: 3, children: 0
opensource
opensource

@lemmy.ml

login for more options
47450
2759
2393

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

Rules

  • Posts must be relevant to the open source ideology
  • No NSFW content
  • No hate speech, bigotry, etc

Related Communities

Community icon from opensource.org, but we are not affiliated with them.

go to feed...