Who does Anubis actually stop?

3 months ago by JensSpahnpasta to c/technik

I have been working on a patch to the Linux kernel to support $ORIGIN for the interpreter (PT_INTERP) via bpf in binfmt_misc [thread].

far_university1990 9 points 3 months ago

For a scraper, solving the Anubis challenge is a one-time, amortized-to-zero cost since the cookie can be cached and reused.

Wait there no request limit per token? The fuck?

Collectively we are wasting an impressive amount of time waiting for access to websites; time we didn’t spend before the AI era. As a human, time is precious and finite to me, whereas to a robot it is not.

If time not matter for robot why all the llm company scrape at site break speed?

The irony is that Anubis’s goal is to stop AI but it was incredibly easy for AI to circumvent it and yet the cost to humans and an open web remains.

Either he not understand POW or never read. Goal is not to stop but to make cost enough to make not worth.

save

path: 0 25002700, hotness: undefined, score: 9, children: 0

DmMacniel 5 points 3 months ago

This is just sad. Instead of understanding WHY a site would employ Anubis they just break it without a care or thought.

save

path: 0 25002092, hotness: undefined, score: 5, children: 1

far_university1990 5 points 3 months ago

I not think they ever broke. They just make a solver for same crypt outside of browser. So just other anubis frontend.

save

path: 0 25002092 25002717, hotness: undefined, score: 5, children: 0

Lemmchen 4 points 3 months ago

It's not designed to stop anybody. It's goal is to slow down automated scrapers.

save

path: 0 25013796, hotness: undefined, score: 4, children: 1

gandalf_der_12te 2 points 3 months ago

automated scrapers of the web UI, specifically. if they use the proper APIs (which are efficient), it's fine.

save

path: 0 25013796 25016339, hotness: undefined, score: 2, children: 0

Sandbar_Trekker 4 points 3 months ago

It's a cat and mouse game.
The more popular something is, the more likely tools will be made to circumvent it.

In this case the author is advertising their tool as a method to circumvent Anubis:

https://github.com/fzakaria/anubis-fetch

save

path: 0 25001378, hotness: undefined, score: 4, children: 1

far_university1990 3 points 3 months ago

This not even circumvent. They just solve anubis challenge like normal. Just in go instead of browser.

save

path: 0 25001378 25002750, hotness: undefined, score: 3, children: 0

gandalf_der_12te 3 points 3 months ago

anubis only stops scrapers that scrap a very large number of the web UI webpages. it does not stop scrapers that act as a fediverse instance and get the data access directly through the API.

the reason why we use it is because scraping the web UI is expensive and slows down the service for everyone else. meanwhile the API access is efficient and therefore not a problem to us.

save

path: 0 25016338, hotness: undefined, score: 3, children: 0