Lemmitors who request subreddits that are useless without the comments, I created a bot that scrapes comments as well

3 years ago by Andreas to c/about

A Reddit-to-Threadiverse archiver for discussion threads. - GitHub - hjalp/leddit: A Reddit-to-Threadiverse archiver for discussion threads.

See the bot in action here!

My instance running Leddit

Click here for a more detailed explanation about the bot's purpose

This bot is intended to be self-hosted. Unfortunately, I can't operate a public instance that takes subreddit requests because of how long syncing comments takes. For comparison, Lemmit takes 21 minutes to sync all of the subreddits on this instance using the old system, but Leddit takes the same amount of time to sync 3 subreddits with around 500k subscribers each once an hour. Smart syncing is planned, but it won't decrease the amount of time taken to sync big and active subreddits.

If you need help setting up an instance, feel free to ask questions in this thread or on the Leddit instance's community.

PixelPassport 3 points 3 years ago

What does this mean for us stupid people? Ooga booga

save

path: 0 760621, hotness: undefined, score: 3, children: 2

Andreas 3 points 3 years ago

Are you asking about what the bot does or how to use this bot?

What it does: The same thing as lemmit.online, but it copies the comments of the original Reddit post to Lemmy as well.

How to use:

Set up a Lemmy instance and make the rate limit very high (I'm happy to make a detailed guide on this if it's needed)
Create an account for the bot
Create a Lemmy community that will store the crossposts
Download the bot's code from Github
Edit the files named .env and config.yaml so the program knows the bot's username, password, what subreddits to fetch content from, and how often
Run the program
Done! Read Reddit posts and comments directly on Lemmy

save

path: 0 760621 763656, hotness: undefined, score: 3, children: 1

PixelPassport 3 points 3 years ago

That's really cool! Thanks for the explanation

save

path: 0 760621 763656 778023, hotness: undefined, score: 3, children: 0