How visible is lemmy on search engine?

3 years ago by axorld to c/asklemmy

I was wondering if the nature of decentralization would negatively affect SEO, since people can access the same post from many different instance

manned_meatball 16 points 3 years ago

https://lemmy.ml/robots.txt , https://lemmy.world/robots.txt , etc don't seem to disallow posts, so the text-based content should be easy to index, at least for these instances.

save

path: 0 239110, hotness: undefined, score: 16, children: 4

Dave_r 8 points 3 years ago

An earlier post pointed out: federated sites seem like they will suffer against central content in a SEO world - regardless of whether they are technically indexable.

I wonder if lemmy should have a SEO friendly federated site... .com domain, robots.txt and everything else...

save

path: 0 239110 239461, hotness: undefined, score: 8, children: 1

manned_meatball 4 points 3 years ago

right, though the SEO game is changing drastically with AI. People are using GPT-like models often in place of searches and likewise, expecting search results to hit their answers rather than being vague pointers. Following this reasoning, the search engines will need direct users to where the valuable information is, not always, but often enough to not lose users to competitors.

So the thing about SEO is that it's often an attention game that advertisers and smaller websites compete with each other. The information in public forums and threads is invaluable for the success of the search engine itself, so they're the ones that will eventually have to adapt to the new federated reality, should it become mainstream - and I do hope so.

save

path: 0 239110 239461 240227, hotness: undefined, score: 4, children: 0

Briskfall 5 points 3 years ago

How long does it usually take for google to index websites? Because I tried the string lemmy site:lemmy.ml after:2023-06-15 and only one post turned up for me and it was Memes... the current state of affairs does not seem promising 😔 And if I tried with another instance with the same keywords lemmy site:kbin.social after:2023-06-15 nothing even turned up.

I wonder though, will search engines adapt to Lemmy and its fediverse system? Or will search engines die? Or will we see dedicated search engines to search through the fediverse?

save

path: 0 239110 240925, hotness: undefined, score: 5, children: 1

manned_meatball 8 points 3 years ago

How long does it usually take for google to index websites?

Anything between a couple of hours to more than a week, I don't think having a "real-time feed" through Google is important though. Other than world cup scores, their results were never about speed.

save

path: 0 239110 240925 243044, hotness: undefined, score: 8, children: 0

AndreTelevise 8 points 3 years ago

The second link for me when searching for Lemmy on Google is the link to the "Join Lemmy" website. Surprisingly, Brave Search, which has seemingly no search bubbles or accounts, shows the same.

save

path: 0 238687, hotness: undefined, score: 8, children: 4

axorld 18 points 3 years ago

Sorry I didnt make the post very clear. I was referring to an individual posts when people search for a specific issue/discussion in Lemmy.

save

path: 0 238687 238936, hotness: undefined, score: 18, children: 1

michikade 3 points 3 years ago

I was googling about Lemmy instances and got several front page results from the Self Hosted community.

save

path: 0 238687 238936 239088, hotness: undefined, score: 3, children: 0

Fleppensteijn 2 points 3 years ago

A couple of days ago Lemmy didn't show in Google Search at all so it's getting better

save

path: 0 238687 245461, hotness: undefined, score: 2, children: 0

bstix 2 points 3 years ago

It'll be a sad day when it reaches first place on Google. Kilmister earned it, but as it is said: a person dies twice. Once when they die, and once again when nobody remembers their name.

save

path: 0 238687 245388, hotness: undefined, score: 2, children: 0

TimothyMcFuck 8 points 3 years ago

Google brings it up

save

path: 0 238825, hotness: undefined, score: 8, children: 0

Spzi 6 points 3 years ago

Oh, good point. Yes, probably? We can not simply assume search engines know that all of these point to the same content:

Or even worse, due to defederation, they may not all point to the exact same content.

Without further investment either from lemmy or the search engine's side, they are probably seen as distinct sources, not aggregated. Which makes each individually less relevant and less likely to show up .

Also note none of the adresses above contain 'lemmy'. How would users search for content on lemmy in these cases? Can't do "technology site:lemmy", or?

But I can say, lemmy content is visible. Haven't seen it on the first page of ecosia yet, but on page 2 or 3.

save

path: 0 251354, hotness: undefined, score: 6, children: 3

Shlomito 2 points 3 years ago

Maybe you could use use site:lemmy.ml, because they federate with most instances, they're likely to have most of lemmy's content?

save

path: 0 251354 307456, hotness: undefined, score: 2, children: 0

phase_change 2 points 3 years ago

Add to that, I’ll bet search engines that find identical content scattered across different sites rank it lower than content at just one site.

save

path: 0 251354 266089, hotness: undefined, score: 2, children: 0

SpliceVW 2 points 3 years ago

This is relatively simple to solve from a technology perspective. You just incorporate the canonical URL meta tag on federated sites that reference the source URL. It'd be trivial to implement, provided the authoritative URL is known.

save

path: 0 251354 289622, hotness: undefined, score: 2, children: 0

BrianTheeBiscuiteer 4 points 3 years ago

I believe there are engines that specifically index (search) the Fediverse. Searx is one that can do this but it might depend on the instance.

save

path: 0 241966, hotness: undefined, score: 4, children: 0

LilyIsWasHere 3 points 3 years ago

I tried searching for the title of this post verbatim and it isn't in google results period.

save

path: 0 241024, hotness: undefined, score: 3, children: 1

beardedrhino 1 point 3 years ago

That could just be because of lag between when the post is created and when the Google crawler finds/indexes the page

save

path: 0 241024 268954, hotness: undefined, score: 1, children: 0

phario 2 points 3 years ago

In my opinion, pushing lemmy to be a Reddit replacement isn’t the right move. Instead, Lemmy should be pushed as something that previously bulletin board forums dominated: the hobbyist, niche, discussion forum.

We don’t need hundreds of millions of users. Hell, most people stopped posting on places on Reddit that had too many users. What you want are communities with thousands of passionate users.

My point is that Google indexing would also be great, but more important is the ability to search these communicators for archivable posts. Reddit’s search functionality was dogshit so we had to rely on Google.

save

path: 0 348164, hotness: undefined, score: 2, children: 1

phario 2 points 3 years ago

“communicators” = “communities”. I can’t seem to find an edit button on Mlem

save

path: 0 348164 348214, hotness: undefined, score: 2, children: 0

Bjoern_Tantau 2 points 3 years ago

Haven't checked if they do this, but you can tell Google which one is supposed to be the "real" post. So you shouldn't get duplicate content.

save

path: 0 240702, hotness: undefined, score: 2, children: 1

AngryClosetMonkey 1 point 3 years ago

The current version of the frontend is not doing that yet, though.