Differences in the sitewide instances blocked between lemmy.ca and sh.itjust.works

4 days ago by SomethingWentWrong to c/lemmy_ca_support

lemmit.online is currently allowed on sh.itjust.works but blocked on lemmy.ca so I looked into it a bit deeper and found there are other differences between the list of sitewide blocked instances for each server.

I'm bringing this to your attention as a low priority background task request to review the differences and see whether the instance in question should be blocked on both sites or allowed for each since they are both fedecan sites.

Here are some stats about the blocked instances:

90 instances are blocked on lemmy.ca
100 instances are blocked on sh.itjust.works
29 blocked instances are common between lemmy.ca and sh.itjust.works
61 instances are blocked only for lemmy.ca but not on sh.itjust.works
71 instances are blocked only for sh.itjust.works but not on lemmy.ca

I'll post in some comments to this post the list of instances blocked on only one of the two fedecan owned lemmy servers and the shell commands I used to gather the data.

Rentlar 4 points 3 days ago

I can tell you one reason for a portion of them on the lemmy.ca exclusive blocks, having been here just a handful of months prior to the Reddit-exodus of June 2023.

Since lemmy.ca is a few years older than sh.itjust.works, there are some servers that were blocked back then, for spammers or trolls that no longer existed by the time sh. first came online. One particularly infamous example from the early Lemmyverse is wolfballs.

Oh and the request/intent to join announcement seemed to indicate that while Fedecan would be an infrastructural and fiscal host for sh., it was agreed the operational management of their Lemmy server would be largely left to the existing people. I suppose no harm in reviewing the differences if there's time to spend, but just trying to provide whatever context I can to your analysis.

(Fixed link)

save

path: 0 25330992, hotness: undefined, score: 4, children: 5

otter 3 points 2 days ago

That is correct :) As of right now, the plan is to keep the federation policy separated at the instance level. There are some subtle differences in the current draft of those policies

Some of the instances in the list are obvious spam/scams that no one wants to see, and sh.it just.works might defederate away from those ones easily. We have only started working together somewhat recently, and its likely that many instances in this category just never federated over to them, or they have already shut down.

For others they will need to raise a discussion with their users to see if that is something the users want. I think it is healthy for there to be some differences in the overall federation lists, since then users can choose which experience is right for them.

Also tagging @SomethingWentWrong@lemmy.ca, thank you for doing the work and sharing those lists / the script. We haven't had a chance to do this kind of analysis yet. Another advantage of listing the federation decisions is that users can compare instances when deciding where to make an account.

save

path: 0 25330992 25349904, hotness: undefined, score: 3, children: 1

SomethingWentWrong 1 point a day ago

My pleasure.

I looked into fediseer and am happy to see it also offers a documented REST API. I compared the blocked and censured list for sh.itjust.works and created this post requesting SJW to to review their mismatches: https://lemmy.ca/post/69562789

FYI - here's current counts of blocked instances and reported censure to fediseer for the top 10 lemmy servers:

                Blocked  Censured
lemmy.blahaj.zone   312       228
lemmy.ca             91         2
lemmy.dbzer0.com    214       158
lemmy.ml            210         0
lemmy.one             4         0
lemmy.world         204       203
lemmy.zip             9         0
programming.dev     324       132
sh.itjust.works     100        94
ttrpg.network         0         0

save

path: 0 25330992 25349904 25363780, hotness: undefined, score: 1, children: 0

SomethingWentWrong 3 points 3 days ago

Thanks for the context.

Your the request/intent to join link doesn't appear to go where it should be.

Email has been doing the "whack-a-mole" battle with spammers for a long time and have tools like https://www.spamhaus.org/blocklists/ to quickly block the latest bad actor.

The Fediverse will likely need to do something similar in identifying and quickly blocking bad actors.

save

path: 0 25330992 25331352, hotness: undefined, score: 3, children: 2

Rentlar 2 points 3 days ago

Sorry bout that, link fixed.

A couple idea threads to pursue in response to your follow ups: Have you seen dbzero's Fediseer? Also, automated community moderation to automatically block users that comment more combatively across other communities and unblock once they establish a pattern of being less so, has been tried on a political news community on lemm.ee before, but lemm.ee is gone now for unrelated reasons. I just wonder if that underlying project is still available and in use somewhere. I don't currently have the energy to look it up.

save

path: 0 25330992 25331352 25331843, hotness: undefined, score: 2, children: 1

SomethingWentWrong 3 points 3 days ago

Ah cool, I wasn't aware of dbzero’s Fediseer. Looks like a good way to identify problematic instances to block.

I'm just an end user of lemmy trying to use it more and went down a rabbit hole about blocked instances when I came across something that impacted me.

save

path: 0 25330992 25331352 25331843 25333350, hotness: undefined, score: 3, children: 0

SomethingWentWrong 2 points 3 days ago

Instances blocked for sh.itjust.works but not lemmy.ca

9kb.me
aethy.com
archaeology.social
beehaw.org
besties.com
bigballchunkyverse.online
bondsdogs.com
botsin.space
bunnyanarchy.org
channels.im
cherryberry.pink
cryptodon.lol
cum.salon
cunnyborea.top
dubvee.org
eda.social
educhat.social
forum-lucifer.com
fun.with.kyun.li
groupsebelah.com
h5q.net
imouto.pics
jenny.nya.pub
kbin.burggit.moe
kernkraft.social
kids.0px.io
kitsui.life
kokuusa.club
kommunismus.social
lawsocial.org
lemmy.byrdcrouse.com
lemmy.fediverse.jp
lemmy.nebtown.info
lemmy.staphup.nl
lolison.top
m.corduba.tech
masost.one
mastadon.astuto.cc
mastodon-ero.xyz
mastodon.integrata-stiftung.de
mastodon.snmsoc.org
mastodon.svgun.ru
mentalhealth-masto.com
mirr0r.city
misskey.doujin.games
mitra.anon-kenkai.com
moar.cachapa.xyz
mstdn.business
mstdn.jp
nasface.cz
neuroscience-mastodon.com
niederbayern.social
nnia.space
owo.town
pedo.school
posting.lolicon.rocks
puls.social
red.computersforpeace.net
redlemmy.com
scfzfilm.org
seda.social
sfw.community
shota.house
social.cutefunny.net
social.tcpcat.net
startrekshitposting.com
tambayan.us
thoughtful.social
transforthe.win
waterlily.tokyo
youjo.love

save

path: 0 25330200, hotness: undefined, score: 2, children: 0

SomethingWentWrong 2 points 3 days ago

shell commands to gather data about blocked instances across multiple lemmy servers

# create an empty directory to work in to avoid clutter
# suggest running this to create a directory for today's date:
DATE=`date +%Y-%m-%d`; mkdir $DATE; cd $DATE

# loop through list of lemmy servers to download json files of blocked instances
# and extract the blocked domains to a txt file
for site in lemmy.ca lemmy.ml lemmy.world lemmy.zip sh.itjust.works
do
  echo $site
  site_base_fname=`echo $site | tr \. _`
  curl -s --request GET \
     --url https://$site/api/v3/federated_instances?kind=blocked \
     --header 'accept: application/json' > blocked_${site_base_fname}.json
  cat blocked_${site_base_fname}.json | jq -r '.federated_instances.blocked.[].domain' | sort -u > blocked_${site_base_fname}.txt
done

# number of common blocked instances for both lemmy.ca and sh.itjust.works
comm -1 -2 blocked_lemmy_ca.txt blocked_sh_itjust_works.txt | wc

# instances only blocked on lemmy.ca but not on sh.itjust.works
comm -2 -3 blocked_lemmy_ca.txt blocked_sh_itjust_works.txt > only_blocked_for_lemmy_ca.txt

# instances only blocked on sh.itjust.works but not on lemmy.ca
comm -1 -3 blocked_lemmy_ca.txt blocked_sh_itjust_works.txt > only_blocked_for_sh_itjust_works.txt

# instances blocked across at least two lemmy servers
# h/t DDG Search Assist AI
awk '{instances[$0]++} END { for (instance in instances) {if(instances[instance]>1) {print instances[instance], instance }}}' blocked*.txt | sort -k1,1nr -k2,2

# instances blocked on at least one lemmy server
awk '{instances[$0]++} END { for (instance in instances) { print instances[instance], instance }}' blocked*.txt | sort -k1,1nr -k2,2

save

path: 0 25330307, hotness: undefined, score: 2, children: 0

SomethingWentWrong 2 points 3 days ago

Instances blocked for lemmy.ca but not sh.itjust.works

alien.top
annihilation.social
api.clubsall.com
baomi.tv
bbs.9tail.net
bbs.darkwitch.net
bolha.social
clatter.eu
clubsall-api.appcafe.workers.dev
clubsall-api.renchesterjramos.workers.dev
clubsall-api2.appcafe.workers.dev
clubsall-api2.renchesterjramos.workers.dev
clubsall-api3.renchesterjramos.workers.dev
clubsall-api4.renchesterjramos.workers.dev
clubsall.com
cubing.social
cum.estate
cunnyborea.space
decayable.ink
demotheque.com
detroitriotcity.com
dot.surf
enterprise.lemmy.ml
eveningzoo.club
fasheng.ing
freeatlantis.com
freespeechextremist.com
gameliberty.club
granitestate.social
http://api.clubsall.com/
lemmit.online
lemmy-mormonsatan-u23030.vm.elestio.app
lemmy.almostadatacenter.social
lemmy.cloudsecurityofficehours.com
lemmy.easfrq.live
lemmy.fedi.bub.org
lemmy.fyi
lemmy.jtmn.dev
lemmy.org
lemmy.podycust.co.uk
lemmy.primboard.de
lemmy.shwizard.chat
lemmy.tedomum.net
mindshare.space
news.deghg.org
nicecrew.digital
nightshift.social
noauthority.social
oceanbreeze.earth
poa.st
pravda.me
rebased.social
rebelbase.site
shitposter.world
thediscussion.site
tummy.town
ussr.win
veenk.help
voyager.lemmy.ml
wolfballs.com
wumbo.buzz

save

path: 0 25330141, hotness: undefined, score: 2, children: 0