At least you got yourself into the contributing mindset. Tackle the next issue!

This is the problem we are having with fast growth on a few select communities. The largest servers are being bogged down simply because the software has not been tuned for these large types of instances yet. ActivityPub works best (in it's current state) by spreading users over smaller/medium sized instances. Folks need to take a look at other instances (and I agree it is hard to find them for a newcomer). You can look at https://fedidb.org/ to look at instances that have been indexed running kbin, lemmy, and other software.

Joining a smaller instance means that your server is not being bogged down by tens of thousands of other users trying to pull updates to their devices at the same time. You can still see the content from other instances, and in many cases it is more reliable because your smaller instance actually has the resources to handle pulling in the posts you want to see. The server-to-server communications that make content federation possible are less resource intensive than pushing updates to user devices. Less users on an instance = more server resources to actually federate content. In the future I am sure instances like lemmy.world will be able to handle the user traffic and federation traffic smoothly, but for now the best way to ensure stability is to join a smaller instance.

(Plug for my instance: https://remy.city, a general purpose Kbin instance. I set it up for personal use but anyone is free to join me in using it. I have defederated from the instances with more extreme viewpoints userbase-wide (like lemmygrad and exploding-heads), and from lemmynsfw.com because of content hosting concerns. I'm open to suggestions on others.)

Your knock on the door analogy is exactly right--when I started my instance, I had to search every community that I wanted to see directly by URL. Then my server would send a message to that community's server saying that I subscribed to that community. Now, every time a post is made at that community, it's server sends my server an update. If I post a comment to a community on lemmy.ca (like I am now), from my kbin instance (remy.city), and you are reading it from kbin.social, that means my server first saved my comment locally, then sent it to lemmy.ca, and lemmy.ca sent it to your kbin.social because you subscribed to the community. So in that case, lemmy.ca is the 'authority', and is responsible for sending updates out to subscribed parties.

There is no such thing for instances--each new instance has to manually make a connection to another (i.e. a user on the new instance must subscribe to something from another instance). I think the tools like fediverse.observer are reading comments or other activity from popular instances, and are then compiling a list of the instances they find by doing that. But there is no central server/authority that makes communication between instances possible. Each instance has to talk to each other instance for it to happen. It's a bit inefficient but is necessary for decentralized communication.

Wikipedia is a good example. It is annoying when they ask for the $3 every year, but it's true that a small contribution like that across the many users can keep a free/libre project sustained. Things like Usenet used to be part of your ISP bill anyhow, so a small monthly/annual amount to your instance host makes sense to me. Of course, we pay ridiculous amounts to our ISPs without services like this nowadays, so it does hurt a little

Sorry, I should have included that info. You got me thinking, however, and I made the decision to defederate from lemmynsfw.com. I'm not against the communities it hosts, but I don't want to deal with any of the content hosting legal questions that come with it (or at least minimize it where I can). There do appear to be some posts that make it in to the 'random' microblog section that are NSFW, I will look into what I can do for those.

There are ways to write links in such a way that they should keep you on your instance, but I'm not too familiar with them. I wonder if it would be possible to "precheck" links that load on a page, and if any point to content that can be federated, kick off the process of pulling that content in. Then when the user clicks that link, it would take them to the content on their home instance, where they can interact. That way users wouldn't need to deal with formatting links a certain way, it would just happen automatically (if your home instance software supports it).

Good idea, I just looked however and they've changed the instances page to: https://fedidb.org/software/kbin. So any server that has federated with any other listed is on there, I believe. More Kbin instances out there than I thought!

Thanks for the great work on this @ernest, and every contributor who's dove in and made a PR.

Good point. Nothing against the larger instance owners of course. If my little instance got super popular somehow (like being recommended in guides on how to join lemmy/kbin), and thousands of users got in per day, I could see issues happening just like this. I don't know the ins and outs of tuning this software for performance at scale, and I know I couldn't learn it fast enough if my instance faced very fast growth like lemmy.world has.

I think admins are going to need to turn registrations off periodically, as they scale their hardware (and their knowledge) to run it for more users effectively.

That might be android only, I'm not sure. I remember seeing a pull request about it having to do with something in the PWA manifest.

I think that would be worth it, yeah. Of course if you are hosting it on your home network there will be some added security concerns (and that might make it better to only allow signups to friends/friends of friends/etc). The way I see it is that some instances are going to host the largest communities, and therefore those instances are going to need to handle all of the incoming/outgoing updates to posts in those communities. Right now they can't do that reliably and push updates out to all of their users' devices.

So in the long run I think having small/medium instances (say a couple hundred, not tens of thousands of users) will be the way to grow. These smaller communities can push updates to their smaller user count reliably, and then have more resources to handle federated content coming in and going out. I think scaling for the incoming/outgoing federation requests would be easier than for direct user activity. Federation stuff can be queued and then spread over time, but user requests cannot be.

Very true, if you don't mind losing your post history. A simple way to migrate subscriptions would be great for those folks that make a new account every once in awhile anyway.

The best way to know who owns the service you use it to own it yourself. That does come with a lot of overhead, though. I started my own kbin instance because I, like you, was worried about "what if one day I go to login and my home instance decided to shut off forever?" That can't happen now, besides if I forget to pay/something goes wrong with the server. But I enjoy tinkering so it works out in my case.

I think ultimately users are responsible for which instance they choose to sign up for. If there isn't much transparency on a certain instance, it may not be the one for you. I agree that the sign up pages could have some areas where more information could be shared. Of course, it is up to each instance admin to share accurate and factual information as to who they are/where the money comes from/goes etc.

Don't they know not to use the window-git AUR package? It's a development build and could be unstable/unsafe.

On Lemmy that is. Kbin I believe replicates everything (unless I set my server up wrong). My server at the moment pulls in around 1.5 GB a day it seems. There is a pull request open on the kbin git repo for a feature to auto-remove old media. Personally I'd like the ability to turn on/off media replication. If an instance wants a complete copy in case of defederation/disconnection somehow, they can opt in and mirror all media that comes in. Most servers should just link to the original image source on the originating instance though.

There is also Kbin (that I use) which can communicate with Lemmy. Since they both share the "sections/communities/groups with threads in them" concept, they can interact pretty well.

I've seen folks calling it the "Threadiverse" because of the different software that servers run.

That'll be slick!

It's not, unfortunately. The main issue I see with implementing that is all of your existing posts/comments would still link to your old account on other instances. So either your old instance would have to forward requests for your account to the new instance, or maybe ActivityPub has some way to push that update out and update your account's home across the fediverse. I don't know enough about it unfortunately.

You are welcome to create multiple accounts in any case. Suppose that defeats the purpose of the fediverse though.

That's a good breakdown, thanks! The bad thing is that I could see these issues happening even unintentionally, with the fact that we have a few large instances vs. many smaller ones. So far we seem to have everyone running the same code, straight from the repositories (at least functionality wise). For my own kbin instance though, I have technically changed things. I changed some code to make a custom logo appear nicely, I've added some padding here and there, etc. I have also thought about implementing an automatic job that clears posts tagged with 'nsfw' or other related things in the microblog feed.

I might implement that, and then submit it to the kbin devs if it works well. There's no guarantee that other admins/devs would do that as well. If they implement a feature that makes their community more popular, they would seem to have incentive to keep it private. And that's where stuff like Meta comes in. If they implement rigorous content filtering, I doubt that would make it into the actual AP protocol. It would be the differentiating factor between using their 'safe' instance, vs. going rugged on an independent instance.

They could say "we implement the ActivityPub protocol as specified" and they wouldn't be wrong. They would just have some extras added onto the top to make their experience more polished. Easy to do when you are a for-profit and have plenty of devs. They would just argue that those are the features that make their interface different, like kbin and lemmy are different.

The only way around it is for communities to agree that they will run the software as released, maybe with only cosmetic changes. Any improvements to functionality should be submitted to the devs so that the wider community can benefit.

You are totally right and my brain definitely farted on that one. Extreme is extreme to me I guess. I'll edit to reflect that.