Lemmy.world status update 2023-07-05

3 years ago by Ruud to c/lemmyworld

Another day, another update.

More troubleshooting was done today. What did we do:

Yesterday evening @phiresky@phiresky@lemmy.world did some SQL troubleshooting with some of the lemmy.world admins. After that, phiresky submitted some PRs to github.
@cetra3@lemmy.ml created a docker image containing 3PR's: Disable retry queue, Get follower Inbox Fix, Admin Index Fix
We started using this image, and saw a big drop in CPU usage and disk load.
We saw thousands of errors per minute in the nginx log for old clients trying to access the websockets (which were removed in 0.18), so we added a return 404 in nginx conf for /api/v3/ws.
We updated lemmy-ui from RC7 to RC10 which fixed a lot, among which the issue with replying to DMs
We found that the many 502-errors were caused by an issue in Lemmy/markdown-it.actix or whatever, causing nginx to temporarily mark an upstream to be dead. As a workaround we can either 1.) Only use 1 container or 2.) set proxy_next_upstream timeout; max_fails=5 in nginx.

Currently we're running with 1 lemmy container, so the 502-errors are completely gone so far, and because of the fixes in the Lemmy code everything seems to be running smooth. If needed we could spin up a second lemmy container using the proxy_next_upstream timeout; max_fails=5 workaround but for now it seems to hold with 1.

Thanks to @phiresky@lemmy.world , @cetra3@lemmy.ml , @stanford@discuss.as200950.com, @db0@lemmy.dbzer0.com , @jelloeater85@lemmy.world , @TragicNotCute@lemmy.world for their help!

And not to forget, thanks to @nutomic@lemmy.ml and @dessalines@lemmy.ml for their continuing hard work on Lemmy!

And thank you all for your patience, we'll keep working on it!

Oh, and as bonus, an image (thanks Phiresky!) of the change in bandwidth after implementing the new Lemmy docker image with the PRs.

Edit So as soon as the US folks wake up (hi!) we seem to need the second Lemmy container for performance. So that's now started, and I noticed the proxy_next_upstream timeout setting didn't work (or I didn't set it properly) so I used max_fails=5 for each upstream, that does actually work.

phiresky 913 points 3 years ago

server load is too low, everyone upvote more stuff so i can optimize more

edit: guess there is some more work to be done 😁

path: 0 849257, hotness: undefined, score: 913, children: 22

woelkchen 148 points 3 years ago

Upvote causes an endless spinner on Liftoff. 😁

path: 0 849257 849952, hotness: undefined, score: 148, children: 4

TurnItOff_OnAgain 44 points 3 years ago

I'm getting 504 gateway time outs when I try to upvote

path: 0 849257 849952 850435, hotness: undefined, score: 44, children: 2

GatoB 18 points 3 years ago

For me it works way better than before

path: 0 849257 849952 850435 852408, hotness: undefined, score: 18, children: 1

TurnItOff_OnAgain 12 points 3 years ago

seems like it may have been a temporary issue. It's clearing back up.

path: 0 849257 849952 850435 852408 852666, hotness: undefined, score: 12, children: 0

mrmanager 1 point 3 years ago

path: 0 849257 849952 1196807, hotness: undefined, score: 1, children: 0

marsokod 118 points 3 years ago

I don't understand your graph. It says you are measuring gigabit/sec but shouldn't the true performance rating be gigabeans/sec for a Lemmy instance?

path: 0 849257 850430, hotness: undefined, score: 118, children: 8

Tosti 36 points 3 years ago

path: 0 849257 850430 852486, hotness: undefined, score: 36, children: 0

ignotum 21 points 3 years ago

And where's the statistics for days between each core dump? A healthy instance should have at least three days between each one

path: 0 849257 850430 852301, hotness: undefined, score: 21, children: 5

rbhfd 12 points 3 years ago

Depends on whether they have fiber or not.

path: 0 849257 850430 852301 853147, hotness: undefined, score: 12, children: 2

Bluetreefrog 4 points 3 years ago

I see what you did there.

path: 0 849257 850430 852301 853147 864743, hotness: undefined, score: 4, children: 0

Zerlyna 1 point 3 years ago

Beans have tons of fiber!!

path: 0 849257 850430 852301 853147 865330, hotness: undefined, score: 1, children: 0

romkube 4 points 3 years ago

path: 0 849257 850430 852301 853100, hotness: undefined, score: 4, children: 0

Zerlyna 1 point 3 years ago

path: 0 849257 850430 852301 865301, hotness: undefined, score: 1, children: 0

unreachable 1 point 3 years ago

gigabean/s

heh

path: 0 849257 850430 879994, hotness: undefined, score: 1, children: 0

Snipe_AT 58 points 3 years ago

aye aye sir, to the upvote machine!

path: 0 849257 849569, hotness: undefined, score: 58, children: 0

PatFussy 28 points 3 years ago

Double the image upload size and you will see more shitposts

path: 0 849257 850111, hotness: undefined, score: 28, children: 1

KluEvo 5 points 3 years ago

I was gonna argue that you'd see more bean posts, but at this point they're the same thing, both in the pun sense and the literal sense

path: 0 849257 850111 852272, hotness: undefined, score: 5, children: 0

s4if 22 points 3 years ago

Web-ui is very smooth rn.. is this .world?
😅
Joke aside, the improvement is like heaven and earth. Love it!. Good work teams!

path: 0 849257 853632, hotness: undefined, score: 22, children: 0

thesanewriter 10 points 3 years ago

I'm on another instance, but here's some federated activity for you.

path: 0 849257 852361, hotness: undefined, score: 10, children: 0

PlutoniumAcid 8 points 3 years ago

All hail @phiresky@lemmy.world! Today is your day. You have made the single most valuable contribution and you must be celebrated! Bravo! Hurrah!

path: 0 849257 859281, hotness: undefined, score: 8, children: 0

sorenant 7 points 3 years ago

I was just going to post a meme about choosing either creating activity or spare the server from overloading. Now the joke won't stick.

path: 0 849257 854216, hotness: undefined, score: 7, children: 0

Tosti 3 points 3 years ago

path: 0 849257 852503, hotness: undefined, score: 3, children: 0

Rootiest 437 points 3 years ago

Test:

Upvote if you can see this comment. 👍

path: 0 853572, hotness: undefined, score: 437, children: 10

jugalator 22 points 3 years ago

Looking good from here.

Edit: And comment rapidly going through. :)

path: 0 853572 853884, hotness: undefined, score: 22, children: 0

Aux 9 points 3 years ago

Can't see anything...

path: 0 853572 856489, hotness: undefined, score: 9, children: 0

CrypticCoffee 3 points 3 years ago

Shame you're getting no karma. Take my upvote.

path: 0 853572 861124, hotness: undefined, score: 3, children: 3

Today 2 points 3 years ago

I think i was on r for a year or two before I learned what karma was. I still don't understand it's value.

path: 0 853572 861124 872945, hotness: undefined, score: 2, children: 0

Rootiest 2 points 3 years ago

Oh, I thought that was the joke 😅

Like a tongue-in-cheek, remember our silly old Reddit ways kinda thing.

path: 0 853572 861124 867548, hotness: undefined, score: 2, children: 1

CrypticCoffee 1 point 3 years ago

Ha. I don't miss it. I upvote for good content.

path: 0 853572 861124 867548 989980, hotness: undefined, score: 1, children: 0

Interesting_Test_814 2 points 3 years ago

I can see and upvote this comment. 👍

path: 0 853572 877834, hotness: undefined, score: 2, children: 1

Rootiest 2 points 3 years ago

I can also see and updoot yours. 👍

path: 0 853572 877834 881351, hotness: undefined, score: 2, children: 0

Speculater 1 point 3 years ago

502, sorry mate.

path: 0 853572 878397, hotness: undefined, score: 1, children: 1

Rootiest 1 point 3 years ago

Damn, better luck next time.

path: 0 853572 878397 881326, hotness: undefined, score: 1, children: 0

isaachernandez 256 points 3 years ago

The change is noticeable. Good job guys.

Thanks for the updates.

path: 0 849163, hotness: undefined, score: 256, children: 3

Carnelian 96 points 3 years ago

I agree. Felt it immediately when I started browsing. Everything is faster and more responsive, on top of the error messages disappearing

path: 0 849163 849333, hotness: undefined, score: 96, children: 1

jake_eric 23 points 3 years ago

Yup I can even post comments first try, without getting an error! Things are working well!

path: 0 849163 849333 851221, hotness: undefined, score: 23, children: 0

FarLine99 2 points 3 years ago

Really noticeable. Cool update. Thank you, guys! ❤️

path: 0 849163 864129, hotness: undefined, score: 2, children: 0

GnothiSeauton 125 points 3 years ago

This is why having a big popular instance isn't all bad. It helps detect and fix the scaling problems and inefficiencies for all the other 1000s of instances out there!

path: 0 851001, hotness: undefined, score: 125, children: 5

AlmightySnoo 60 points 3 years ago

This, if everyone kept just spreading out to smaller instances as suggested in the beginning, while still a sensible thing to do, no one would have noticed these performance issues. We need to think a few years out, assuming Lemmy succeeds and Reddit dies, and expect that "small instance" will mean 50k users.

path: 0 851001 851803, hotness: undefined, score: 60, children: 1

deweydecibel 14 points 3 years ago

I sincerely doubt reddit will die anytime soon, it'll just exist as its own thing that it's new target audience gets bored with and moves on from in a few years when something new and flashy catches their eye in the app store. Just like they do all the other apps designed in exactly the same fashion that reddit is currently morphing into.

Meanwhile Lemmy will be slowly building it's communities up to be what reddit used to be.

path: 0 851001 851803 864192, hotness: undefined, score: 14, children: 0

bilb 18 points 3 years ago

I'm actually kinda waiting a few releases to start promoting my instance anywhere, letting some other brave instance admins work the kinks out a bit first.

path: 0 851001 852499, hotness: undefined, score: 18, children: 0

rockstarpirate 14 points 3 years ago

Agreed. I decided to keep my community on lemmy.world specifically because of the community investment I see being put into it.

path: 0 851001 852853, hotness: undefined, score: 14, children: 0

avidamoeba 6 points 3 years ago

If this project is to stay for the long haul, we gotta load test it and stabilize it. These folks are doing the important work here. Large instances are more or less inevitable if Lemmy sticks.

path: 0 851001 854865, hotness: undefined, score: 6, children: 0

mintiefresh 124 points 3 years ago

Wow. So much smoother today.

Great work.

You dropped this 👑

path: 0 867475, hotness: undefined, score: 124, children: 2

Boiglenoight 13 points 3 years ago

Damn tootin.

path: 0 867475 869610, hotness: undefined, score: 13, children: 1

Krompus 8 points 3 years ago

You're gonna be when you see how fast I can bean post now.

path: 0 867475 869610 870447, hotness: undefined, score: 8, children: 0

kionay 115 points 3 years ago

You guys had better quit it with all this amazing transparency or it's going to completely ruin every other service for me. Seriously though amazing work and amazing communication.

path: 0 875161, hotness: undefined, score: 115, children: 0

dreadedsemi 101 points 3 years ago

My upvote can go through fast now

Good work

path: 0 870020, hotness: undefined, score: 101, children: 0

wilberfan 99 points 3 years ago

I love the smell of updates in the morning.

path: 0 849211, hotness: undefined, score: 99, children: 0

sv1sjp 92 points 3 years ago

Thank you guys for your awesome work!

Also to other people: DONATE TO FOSS PROJECTS. If 50.000 people donate only 0.5€, we have 25.000€ for funding the servers, coding, motivating/ people etc. Just don't take a cup of coffee for 1 day. We are already 2 millions in Lemmy instances. We can build a decentralized world together!!

path: 0 862094, hotness: undefined, score: 92, children: 6

wmrch 24 points 3 years ago

You can pry my cup of coffee from my my cold, dead hands.

Will donate anyway, I really want this project to keep going.

path: 0 862094 864175, hotness: undefined, score: 24, children: 0

bappity 20 points 3 years ago

euro contributed 🫡

path: 0 862094 863997, hotness: undefined, score: 20, children: 0

CaptPretentious 7 points 3 years ago

Is there a good link? One for USD?

path: 0 862094 865097, hotness: undefined, score: 7, children: 1

adamantris 3 points 3 years ago

If there's a thing you really need to pay in a foreign currency, look into Revolut or Wise. Since I occasionally have to pay stuff in Turkish lira, GBP and donated to USD-only Receivers I like to keep Revolut as my secondary bank account since exchanging one currency to another is completely free!

path: 0 862094 865097 867533, hotness: undefined, score: 3, children: 0

PropaGandalf 7 points 3 years ago

you inspired me to serve the greater good!

path: 0 862094 863516, hotness: undefined, score: 7, children: 1

sv1sjp 18 points 3 years ago

For example, if you speak a second language, you can even help with translation in projects. Its very easy. E.g. I translated the Jerboa (Lemmy client for Android) in Greek 2-3 days ago. I needed only 1 hour to finish and special 15-20 minutes for fixes that I missed, yesterday.

path: 0 862094 863516 863910, hotness: undefined, score: 18, children: 0

Marxine 81 points 3 years ago

Boy does it feel good to have those reports and understand the work you guys do. It's really inspiring! Thanks for your hard work, everything has been silk smooth! This instance is really great, Lemmy and its devs are really amazing and I feel at home in a nice, cozy community.

path: 0 865172, hotness: undefined, score: 81, children: 0

Spectator 80 points 3 years ago

I'm not sure wtf you just said, but lemmy.world feels very smooth today, so thank you for your continued hard work!

path: 0 874102, hotness: undefined, score: 80, children: 0

sirnak 79 points 3 years ago

Am I getting this correct: the whole lemmy.world instance run in one single container on one single host?

path: 0 864039, hotness: undefined, score: 79, children: 5

cley_faye 47 points 3 years ago

You'd be surprised at how much performance this kind of setup can squeeze off. Often the limitation is more on the DB/storage than network handling and processing power.

path: 0 864039 865399, hotness: undefined, score: 47, children: 1

eek2121 24 points 3 years ago

This. Most of the time, the bottleneck will be the database backend.

Curious if lemmy.world uses separate reader/writer instances.

path: 0 864039 865399 867177, hotness: undefined, score: 24, children: 0

ABC123itsEASY 23 points 3 years ago

Impressive if true!

path: 0 864039 864355, hotness: undefined, score: 23, children: 0

bappity 16 points 3 years ago

if it runs this well on a single container, considering the amount of users it has, I fear the power it'd hold with more

path: 0 864039 866429, hotness: undefined, score: 16, children: 0

tempest 10 points 3 years ago

Lemmy is built on an async work stealing runtime and you can get very large instances from Amazon and the like.

I'm sure the instance isn't massive but number of containers and a single "host" are not great indicators of efficiency.

path: 0 864039 865470, hotness: undefined, score: 10, children: 0

Timou 69 points 3 years ago

So that's why it was so smooth today... Great work!

path: 0 861639, hotness: undefined, score: 69, children: 1

erik1984 25 points 3 years ago

path: 0 861639 862234, hotness: undefined, score: 25, children: 0

ericjmorey 65 points 3 years ago

Submitting PRs is literally the most effective response that helps everyone who uses Lemmy. Thanks to you all.

path: 0 849386, hotness: undefined, score: 65, children: 0

MR_GABARISE 63 points 3 years ago

This is better optimization than most enterprise devs will see in their lifetimes.

path: 0 870553, hotness: undefined, score: 63, children: 3

xandertron 61 points 3 years ago

upvoting posts is so much more stable now, we might actually see more bean posts as a result

path: 0 859239, hotness: undefined, score: 61, children: 0

MindfuckRocketship 58 points 3 years ago

Upvotes are still getting rejected. Replies hang so I cancel out and it turns out they did post.

That said, browsing is pretty snappy and smooth. I know the kinks will get worked out eventually. Thanks for the update.

Edit: This now appears resolved minutes later. All smooth on my end.

path: 0 850673, hotness: undefined, score: 58, children: 9

ruud 94 points 3 years ago

Hmm. Seems to work for me.. (Yes this is a test reply)

path: 0 850673 850785, hotness: undefined, score: 94, children: 5

G_Wash1776 31 points 3 years ago

Everyone it’s a test reply, deploy the upvotes

path: 0 850673 850785 851254, hotness: undefined, score: 31, children: 2

sorenant 9 points 3 years ago

I'm going to take the opportunity to test a reply as well.

path: 0 850673 850785 851254 854150, hotness: undefined, score: 9, children: 1

boeman 6 points 3 years ago

It's a good thing the points mean even less on Lemmy.

path: 0 850673 850785 851254 854150 855478, hotness: undefined, score: 6, children: 0

MindfuckRocketship 9 points 3 years ago

I’ve since successfully upvoted some comments and made replies without it hanging with the spinning circle. Not sure what the issue was but it all seems to be running smoothly now. Thanks.

path: 0 850673 850785 850999, hotness: undefined, score: 9, children: 1

Limeey 4 points 3 years ago

I was getting upvote errors earlier, too, but now it seems smoothed out.

path: 0 850673 850785 850999 851265, hotness: undefined, score: 4, children: 0

ebits21 6 points 3 years ago

I’ve also had errors posting but the reply still goes through.

path: 0 850673 851451, hotness: undefined, score: 6, children: 0

FlaxPicker 4 points 3 years ago

Test comment …

path: 0 850673 851418, hotness: undefined, score: 4, children: 1

FlaxPicker 2 points 3 years ago

Not getting the upvote or comment hang anymore so far.

path: 0 850673 851418 851432, hotness: undefined, score: 2, children: 0

teraflopsweat 53 points 3 years ago

path: 0 864344, hotness: undefined, score: 53, children: 0

Molecular0079 53 points 3 years ago

This is why I love open source. The fact that a community can directly debug the code that's it's being hosted on and directly contribute the improvements back is just wild. Thanks for all the hard work @ruud@lemmy.world and the rest of the lemmy.world team! The site already feels much more responsive.

path: 0 854669, hotness: undefined, score: 53, children: 0

MrPoopyButthole 49 points 3 years ago

The server is absofuckinglutely flying today! It feels smooth and bug free!!! You guys are legends.

path: 0 857239, hotness: undefined, score: 49, children: 0

_Rho_ 46 points 3 years ago

As a data engineer, I'd be interested in hearing more about the SQL troubleshooting.

EDIT: It looks like !lemmyperformance@lemmy.ml is a good place to subscribe to for more technical info on some of these performance improvements.

Also the Lemmy GitHub of course contains more information on bugs/enhancements/etc.

path: 0 868128, hotness: undefined, score: 46, children: 5

tsz 5 points 3 years ago

Yes please

path: 0 868128 871318, hotness: undefined, score: 5, children: 0

Schmedes 4 points 3 years ago

Same, my job is like 80% SQL, so it'd be cool to see what is used in the background and maybe help improve things.

path: 0 868128 912887, hotness: undefined, score: 4, children: 0

Dick_Justice 3 points 3 years ago

Me too!

path: 0 868128 872042, hotness: undefined, score: 3, children: 0

DeputyRan 1 point 3 years ago

Same here! If you find a way to contribute please give us an update!

path: 0 868128 1037932, hotness: undefined, score: 1, children: 1

_Rho_ 1 point 3 years ago

Just edited my comment!

path: 0 868128 1037932 1047420, hotness: undefined, score: 1, children: 0

BitOneZero 40 points 3 years ago

Good to see a heavy production server taking on the scaling issues. Thank you! To discuss Lemmy performance issues, there is a dedicated community: !lemmyperformance@lemmy.ml

path: 0 849389, hotness: undefined, score: 40, children: 0

0235 38 points 3 years ago

Appreciate that these updates use the yyyy-mm-dd format :D

path: 0 894134, hotness: undefined, score: 38, children: 4

asamson23 36 points 3 years ago

It now feels pretty good to browse and it now makes the experience of using Lemmy much more enjoyable. Having to spam the vote buttons was really annoying.

path: 0 853291, hotness: undefined, score: 36, children: 1

MrPoopyButthole 4 points 3 years ago

The vote button change was also the one I immediately noticed!

path: 0 853291 857318, hotness: undefined, score: 4, children: 0

JoeKrogan 35 points 3 years ago

Thanks to all involved across the board. Great work all around 👏👏

path: 0 849233, hotness: undefined, score: 35, children: 0

xavier666 34 points 3 years ago

Even though i'm not from this instance, this is such a nice way of keeping the users posted about changes. I wish more companies (I know this is not a company) went straight to the point, instead of using vague terms like "improved stability, fixed few issues with an update" when things are changed. I hope all instance owners follow this trend.

path: 0 881826, hotness: undefined, score: 34, children: 3

radfordhound 34 points 3 years ago

It's so smooth now; the speed difference is insane! You all are doing excellent work!

path: 0 874329, hotness: undefined, score: 34, children: 0

nostalgicgamerz 34 points 3 years ago

Can we have an update on the status of Lemmy.world and how close ties we are going to have with Meta's threads? Threads is going to support ActivityPub, but time has shown that this is an attempt to try to kill this open platform and eventually replace it with theirs once they get everyone in their ecosystem. (Embrace, Extend...extinguish) Mastodon has said today that they don't mind sleeping with vipers when their demise / dissolution is in Meta's best interest.

Please tell me we are defederating from Meta....or let us know what to expect

EDIT: I originally stated that Mastodon told them to fuck off, but I got confused with Fosstodon (who did that). Mastodon doesn't mind being in bed with Meta

path: 0 869148, hotness: undefined, score: 34, children: 7

balance_sheet 4 points 3 years ago

path: 0 869148 872467, hotness: undefined, score: 4, children: 3

Nilex 4 points 3 years ago

Where have you seen Mastodon formally state they have no interest in working with them?

I'm genuinely asking because I'm relatively new to Mastodon and Lemmy and want to be as informed as possible with this whole Meta situation. And just to be transparent, I'm not a Meta fan at all, to the extent I've never had an account with any of their products

I did read this official Mastodon blog post today...

https://blog.joinmastodon.org/...

and my question to you is because I'm not seeing as quite aggressive a stance as you mentioned.

path: 0 869148 872205, hotness: undefined, score: 4, children: 1

Nilex 1 point 3 years ago

path: 0 869148 872131, hotness: undefined, score: 1, children: 0

helios 32 points 3 years ago

Thanks for this very nice report.

path: 0 849158, hotness: undefined, score: 32, children: 0

simple 31 points 3 years ago

Huge props to everyone working on the project. It's awesome seeing everyone work together and resolving issues so quickly!

path: 0 849475, hotness: undefined, score: 31, children: 0

yonerboner 29 points 3 years ago

Thanks for the detailed update and all the hard work you guys are doing!

path: 0 867033, hotness: undefined, score: 29, children: 0

ekZepp 28 points 3 years ago

path: 0 902367, hotness: undefined, score: 28, children: 0

solidgrue 28 points 3 years ago

Gadzooks! These are huge fixes. Compliments to the team, you guys pulled off a small miracle today.

path: 0 872610, hotness: undefined, score: 28, children: 0

Marxine 26 points 3 years ago

Lemmy's devs and the .world admins have done in a month what Reddit hasn't done in it's whole existence: having a smooth and almost bug-free experience.

Jerboa feels so damn FRESH to use now!

path: 0 865376, hotness: undefined, score: 26, children: 2

wolfpack86 19 points 3 years ago

Not to undervalue the efforts going into this, because I appreciate the new community and the transparency, but I believe we have wildly different definitions of 'almost bug free'

Which, is also something to consider about user experience consistency. Will be a challenge with growth. Fortunately, plugged in admins and devs will help.

path: 0 865376 866598, hotness: undefined, score: 19, children: 1

Marxine 6 points 3 years ago

By almost bug free I was speaking only about my own experience. There are probably loads of things under the hood I'm not noticing, but it's been hours since I last noticed any issue.

I agree that we've still got a long way to go though: both Lemmy and Jerboa are far from their 1.0 release yet.

path: 0 865376 866598 868470, hotness: undefined, score: 6, children: 0

Oxff 25 points 3 years ago

The site is running so much better now, thanks to all. BTW: Love these updates!

path: 0 859680, hotness: undefined, score: 25, children: 0

Jaxia 25 points 3 years ago

I don't understand anything other than you worked diligently to make things smoother. Thanks to everyone for their wonderful work!

path: 0 849483, hotness: undefined, score: 25, children: 1

SomeoneElse 2 points 3 years ago

Same! My first thought was “that’s an impressing looking graph. I have absolutely no idea what it means.” The proof is in the pudding though - lemmy.world is much improved!

path: 0 849483 855481, hotness: undefined, score: 2, children: 0

MetricExpansion 24 points 3 years ago

I'm very curious: does single Lemmy instance have the ability to horizontally scale to multiple machines? You can only get so big of a machine. You did mention a second container, so that would suggest that the Lemmy software is able to do so, but I'm curious if I'm reading that right.

path: 0 876553, hotness: undefined, score: 24, children: 6

KSPAtlas 23 points 3 years ago

Shouldn't the correct HTTP status code for a removed API be 410? 404 indicates the domain wasn't found or doesn't exist, 410 indicates a resource being removed

path: 0 879839, hotness: undefined, score: 23, children: 3

Kodiack 22 points 3 years ago

Awesome work - things seem to be running much more smoothly today.

Do you have anything behind CDN by chance? Looking at the lemmy.world IPs, the server appears to be hosted in Europe and web traffic goes directly there? IPv4 apparently seems to be resolving to a Finland-based address, and IPv6 apparently seems to be resolving to a Germany-based address.

If you put the site behind a CDN, it should significantly reduce your bandwidth requirements and greatly drop the number of requests that need to hit the origin server. CDNs would also make content load faster for people in other parts of the world. I'm in New Zealand, for example, and I'm seeing 300-350 ms latency to lemmy.world currently. If static content such as images could be served via CDN, that would make for a much snappier browsing experience.

path: 0 879029, hotness: undefined, score: 22, children: 4

dyslexicdainbroner 22 points 3 years ago

How great is it to be a part of history in the making -

This is Web 3 in its fomenting -

Headlines ~5yrs:

The ending of Web 2 was unceremonious and just ugly. u/spez and moron@musk watched as their social media networks signaled the end of Web 2 and slowly dissolved. Blu bird’s value disintegrated and Reddit’s hopes for IPO did likewise. Twitter and Reddit dissolved into odorous flatulence as centralization fell apart to the world’s benefit. Decentralized/federated social media such as Mastodon and Lemmy made their convoluted progress and led Web 3’s development and growth…

This is how history is made, it’s ugly and convoluted but comes out sweeet…

path: 0 875997, hotness: undefined, score: 22, children: 0

pathief 21 points 3 years ago

Is it safe to use 2FA yet?

path: 0 881784, hotness: undefined, score: 21, children: 8

httperror418 21 points 3 years ago

Whilst I'm aware that too many users on one instance can be a bad thing for the wider Fediverse, I think it is a great thing at the moment in terms of how well people are banding together to fix the issues being encountered from such a surge in users.

The issues being found on lemmy.world results in better lemmy instances for everyone and improves the whole Fediverse of lemmy instances.

I'm very impressed with how well things are being debugged under pressure, well done to all those involved 👏

path: 0 863414, hotness: undefined, score: 21, children: 1

Molecular0079 2 points 3 years ago

I agree, it's great to see Lemmy being battle-tested in large instance scenarios.

path: 0 863414 867302, hotness: undefined, score: 2, children: 0

shotgun_crab 18 points 3 years ago

You guys are absolute legends, thanks for the update!

path: 0 881941, hotness: undefined, score: 18, children: 0

lwuy9v5 18 points 3 years ago

That's so awesome! Look at that GRAPH!

I'd volunteer to be a technical troubleshooter - very familiar with docker/javascript/SQL, not super familiar with rust - but I'm sure yall also have an abundance of nerds to lend a hand.

path: 0 875007, hotness: undefined, score: 18, children: 1

Contravariant 18 points 3 years ago

Hey I can upvote now!

path: 0 868281, hotness: undefined, score: 18, children: 0

cani 17 points 3 years ago

path: 0 868121, hotness: undefined, score: 17, children: 0

DelvianSeek 16 points 3 years ago

You guys are absolutely amazing. So many thanks to you @Ruud and the entire admin/troubleshooting team! Thank you.

path: 0 876181, hotness: undefined, score: 16, children: 0

CIA_chatbot 16 points 3 years ago

It blows my mind with the amount of traffic you guys must be getting that you are only running one container and not running in a k8s cluster with multiple pods (or similar container orchestration system)

Edit: misread that a second was coming up, but still crazy that this doesn’t take some multi node cluster with multiple pods. Fucking awesome

path: 0 890342, hotness: undefined, score: 16, children: 4

Puzzlehead 15 points 3 years ago

smoooooooooth! Keep up the good work!

path: 0 858574, hotness: undefined, score: 15, children: 0

nuzzlerat 13 points 3 years ago

Is it weird that I’m always excited to read the update posts?

path: 0 849884, hotness: undefined, score: 13, children: 1

thesanewriter 7 points 3 years ago

Not at all. The admins here are doing great work and their updates are often informative and helpful, it makes sense you'd look forward to them.

path: 0 849884 852448, hotness: undefined, score: 7, children: 0

Anti_Weeb_Penguin 13 points 3 years ago

Installed Jerboa again and it feels smoother than Reddit itself, great job!

path: 0 863406, hotness: undefined, score: 13, children: 1

sv1sjp 3 points 3 years ago

It reminds me somehow thw old Google+ in the mid 2010s

path: 0 863406 863931, hotness: undefined, score: 3, children: 0

InfiniteVariables 13 points 3 years ago

Wow it is smooth as butter now. Great job ruud and team!

path: 0 860150, hotness: undefined, score: 13, children: 0

StringyCheese 12 points 3 years ago

@ruud crazy impressive

path: 0 865210, hotness: undefined, score: 12, children: 1

Enrico 4 points 3 years ago

Agreed

path: 0 865210 865460, hotness: undefined, score: 4, children: 0

EvilCartyen 12 points 3 years ago

Things have been super smooth lately, thanks for all the work!

path: 0 849379, hotness: undefined, score: 12, children: 1

G_Wash1776 4 points 3 years ago

Yeah this morning everything has loaded so much quicker, I’ve been able to post and vote on comments no problem! Lemmy is really starting to take form. I fucking love this whole thing.

path: 0 849379 851242, hotness: undefined, score: 4, children: 0

MiddleWeigh 11 points 3 years ago

I took a SM break for a few days, and it's running noticeably better today...I think. (:

Thanks a bunch for floating us degenerates.

path: 0 902875, hotness: undefined, score: 11, children: 0

Zrob 11 points 3 years ago

Awesome work. Any way other devs can contribute?

path: 0 857117, hotness: undefined, score: 11, children: 5

MrPoopyButthole 10 points 3 years ago

Go checkout the Github issues for Lemmy

path: 0 857117 857283, hotness: undefined, score: 10, children: 4

Zrob 7 points 3 years ago

Already on it sir

Just seeing if there’s anything else

path: 0 857117 857283 857365, hotness: undefined, score: 7, children: 3

Zenbach 3 points 3 years ago

You're a good person.

path: 0 857117 857283 857365 858326, hotness: undefined, score: 3, children: 0

Coelacanth 3 points 3 years ago

Keep tabs on !lemmyperformance@lemmy.ml as well if you're not already subscribed to it.

path: 0 857117 857283 857365 860000, hotness: undefined, score: 3, children: 1

Zrob 2 points 3 years ago

Thanks!

path: 0 857117 857283 857365 860000 878942, hotness: undefined, score: 2, children: 0

WolfhoundRO 11 points 3 years ago

Really great job, guys! I know from my experience in SRE that these types of debugs, monitoring and fixes can be much pain, so you have all my appreciation. I'm even determined to donate on Patreon if it's available

path: 0 881802, hotness: undefined, score: 11, children: 1

Tygr 11 points 3 years ago

It felt like I’d jinx us all if I commented but THANK YOU! This has been a wonderful experience today. Absolutely loving it and knew you just needed some time to work out the kinks that happen with fast growth.

path: 0 867513, hotness: undefined, score: 11, children: 0

Datzevo 11 points 3 years ago

You know there's something about dealing with the lagginess in the past few days makes me appreciate the fast and responsive of the update. It nice to see the community grows and makes the experience at Lemmy feels authentic.

path: 0 873185, hotness: undefined, score: 11, children: 0

bathorygod 10 points 3 years ago

You guys did an awesome job.

path: 0 864195, hotness: undefined, score: 10, children: 0

slashzero 10 points 3 years ago

As a Performance Engineer myself, these are the kind of performance improvements I like to see. Those graphs look wonderful. Nice job to all.

path: 0 873008, hotness: undefined, score: 10, children: 0

subspaceinterferents 9 points 3 years ago

thankyouthankyouthankyouthankyouthankyou it's working very well today...

path: 0 869118, hotness: undefined, score: 9, children: 0

somedaysoon 9 points 3 years ago

path: 0 855611, hotness: undefined, score: 9, children: 1

Zenbach 2 points 3 years ago

It's smooth as butter today

path: 0 855611 858338, hotness: undefined, score: 2, children: 0

CriticalMiss 9 points 3 years ago

I like that the post goes in detail and allows us tech nerds to get hard watching this stuff instead of the regular corpo jumbo change log that consists of:

we uhh fixed some stuff so yeah good?

path: 0 853311, hotness: undefined, score: 9, children: 0

BURN 9 points 3 years ago

I love reading these! Thanks for all the work

I hope to start on some small contributions sometime next week. Stability has been noticeably better the last few days and I imagine it’s only going to get better.

path: 0 859878, hotness: undefined, score: 9, children: 0

IanM32 9 points 3 years ago

A lot of this stuff is pretty opaque to me, but I still read through it because I love how detailed they are in sharing what's going on under the hood and how it relates to problems users experience. Kudos to you guys!

path: 0 873727, hotness: undefined, score: 9, children: 0

ef9357 9 points 3 years ago

Love the transparency. Thanks to the entire team!

path: 0 909801, hotness: undefined, score: 9, children: 0

anal_enjoyer 9 points 3 years ago

Love these updates! Love the transparency!

path: 0 916722, hotness: undefined, score: 9, children: 0

DharkStare 8 points 3 years ago

Great job. Everything seems to be working smoothly for me now. The past several days have been a bit rough but now it's all working.

path: 0 893671, hotness: undefined, score: 8, children: 0

Olap 8 points 3 years ago

Proving that orgs do need DBAs!

path: 0 850192, hotness: undefined, score: 8, children: 0

KonQuesting 8 points 3 years ago

Thanks for the updates! Seeing the details of how you work through these early issues is valuable to those of us thinking of starting an instance.

path: 0 929475, hotness: undefined, score: 8, children: 0

Hynek29 8 points 3 years ago

Great job guys! It really feels more responsive today

path: 0 880392, hotness: undefined, score: 8, children: 0

MarsAgainstVenus 8 points 3 years ago

Definitely noticeable!

Also, the amount of communication here is AWESOME! Thank you for keeping us updated.

path: 0 849744, hotness: undefined, score: 8, children: 0

Boiglenoight 7 points 3 years ago

Compared to days prior, things are running much better today. Page load speed, reliable post/reply/upvoting. I dunno if it's just happenstance but whatever knobs, levers and keystrokes you're manipulating, keep going the things! Thanks so much for a home absent of corporate BS.

path: 0 855753, hotness: undefined, score: 7, children: 0

Graphine 7 points 3 years ago

Man I thought I noticed something different. For the past week or so I've gotten nothing but network error and Java errors in Jerboa which are completely gone now. Posts load almost instantly too. Appreciate the effort guys, was going insane.

path: 0 865712, hotness: undefined, score: 7, children: 0

Techie 7 points 3 years ago

Awesome, thanks!

path: 0 849370, hotness: undefined, score: 7, children: 0

atretador 7 points 3 years ago

Ty for the hard work <3

path: 0 869272, hotness: undefined, score: 7, children: 0

NatoBoram 7 points 3 years ago

It works so well, that's very refreshing

path: 0 875760, hotness: undefined, score: 7, children: 0

lichkain 7 points 3 years ago

Would HAProxy work better as a load balancer? For work we switched due to some issues with NGINX; so far, the service has been much more consistent with pretty much no downtime, even when restarting server hosts.

path: 0 875652, hotness: undefined, score: 7, children: 1

legroom 7 points 3 years ago

Great work !!

path: 0 862267, hotness: undefined, score: 7, children: 0

Ipodjockey 7 points 3 years ago

Sweet. Yeah it is definitely snappier today. Feels great.

path: 0 852244, hotness: undefined, score: 7, children: 0

tiwenty 7 points 3 years ago

I agree that everything is a lot more stable!

path: 0 850008, hotness: undefined, score: 7, children: 0

Izzy 6 points 3 years ago

The responsiveness of the website has felt drastically better today.

path: 0 854689, hotness: undefined, score: 6, children: 0

sunnyxiongster 6 points 3 years ago

Everytime I open a post and go back to previous page it scrolls back to top. Is this fixable? Im on windows 11, chrome.

path: 0 879638, hotness: undefined, score: 6, children: 2

throwsbooks 6 points 3 years ago

Now that's a satisfying couple of charts!

path: 0 850930, hotness: undefined, score: 6, children: 0

pickledredonions 6 points 3 years ago

Amazing work, thank you so much!

path: 0 882329, hotness: undefined, score: 6, children: 0

Pro75357 6 points 3 years ago

It's definitely a lot better today. Great work.

path: 0 867271, hotness: undefined, score: 6, children: 0

jcg 6 points 3 years ago

Wow I applied these PRs on my server as well, running waaay lighter now. And it seems the federation misses have cleared up! Bravo Lemmy.world team!

path: 0 923723, hotness: undefined, score: 6, children: 0

NegativeCool 6 points 3 years ago

This was cool to read.

path: 0 876377, hotness: undefined, score: 6, children: 0

RxBrad 6 points 3 years ago

path: 0 855780, hotness: undefined, score: 6, children: 0

Sekemoto 6 points 3 years ago

It is much smoother than it was previously. Thank you!

path: 0 874854, hotness: undefined, score: 6, children: 0

oryx 6 points 3 years ago

Things have been very noticably improving every day since the big update! I've had very, very few problems today browsing on Jerboa.

path: 0 856370, hotness: undefined, score: 6, children: 0

Drunemeton 6 points 3 years ago

Thanks for the update and thanks for the amazing collaboration you’re fostering!

I feel like Lemmy.World is leading the way for other instances to follow, and that’s a good thing to do. Even if it’s painful for everyone involved.

path: 0 856407, hotness: undefined, score: 6, children: 0

ramblechat 6 points 3 years ago

Seems a lot faster today - great work!

path: 0 919299, hotness: undefined, score: 6, children: 1

isdfoa 6 points 3 years ago

Awesome, no more errors on my end! Thanks for the hard work, donated as a small token of thanks for the team here.

path: 0 865256, hotness: undefined, score: 6, children: 0

evil_opossum 5 points 3 years ago

I'll be honest I don't know what any of this means but what I can say is I absolutely love the transparency of all of this. It's so refreshing and maybe I'll start learning more about what I'm looking at because I'll keep seeing it. Great work!

path: 0 852106, hotness: undefined, score: 5, children: 0

sorenant 5 points 3 years ago

Ever since I joined Lemmy I had problems posting replies but that kink seems to have been ironed out. Thanks!

path: 0 854285, hotness: undefined, score: 5, children: 0

Spliffman1 5 points 3 years ago

path: 0 873148, hotness: undefined, score: 5, children: 0

BendyLemmy 5 points 3 years ago

ROFL Back in the day, when AOL was a thing - I used to think it stood for 'Assholes Online'.

Good luck (",) appreciate the efforts you make fighting the crazy deluge.

path: 0 871774, hotness: undefined, score: 5, children: 0

Denuath 5 points 3 years ago

This update is a difference like night and day. Very impressive!

path: 0 866040, hotness: undefined, score: 5, children: 0

Wet 5 points 3 years ago

Thank you guys, you are doing an amazing job. These are great times for the internet and you are a huge part of it!

path: 0 863391, hotness: undefined, score: 5, children: 0

CreativeCider 5 points 3 years ago

Well done guys!

path: 0 863587, hotness: undefined, score: 5, children: 0

coconutxyz 5 points 3 years ago

path: 0 850898, hotness: undefined, score: 5, children: 0

ikidd 5 points 3 years ago

My god, it's fast right now. Don't touch anything.

Good job.

path: 0 861999, hotness: undefined, score: 5, children: 1

subspaceinterferents 1 point 3 years ago

"Don't touch anything." lol

path: 0 861999 869210, hotness: undefined, score: 1, children: 0

zacher_glachl 5 points 3 years ago

Thank you for all your great work, and I really appreciate and enjoy the updates about the gritty details!

path: 0 860748, hotness: undefined, score: 5, children: 0

srasmus 5 points 3 years ago

Was just noticing how much smoother it is this morning. Great work!

path: 0 850847, hotness: undefined, score: 5, children: 0

snowe 5 points 3 years ago

@ruud@lemmy.world is this docker container y'all are using available on a registry? We'd like to use it. And do you have a load balancer in front of your lemmy-ui image to allow two containers to run? or is that built in and I just never noticed it?

path: 0 860607, hotness: undefined, score: 5, children: 1

ruud 10 points 3 years ago

Well we use the cetra3/lemmy:the-phiresky-cut image but you can also wait for the next RC or release which will have the PRs. We load balance with nginx which works if you use max_fails=5 for each upstream.

path: 0 860607 862844, hotness: undefined, score: 10, children: 0

wildeaboutoskar 5 points 3 years ago

I'll be honest, I think I understood three words of your OP. Sounds like the general gist though is Woo so great 😃

Thanks for all your efforts, glad someone knows what they're doing

path: 0 859074, hotness: undefined, score: 5, children: 0

Shadywack 5 points 3 years ago

It's really fun to see these updates, and get insight into what you're putting into this instance. It's real obvious that it's a labor of love.

path: 0 851993, hotness: undefined, score: 5, children: 0

SapienSRC 4 points 3 years ago

Thank you for all the hard work and transparency as always! Everything is running perfectly knocks on wood

path: 0 875756, hotness: undefined, score: 4, children: 0

anthonium 4 points 3 years ago

Thank you so much for all the efforts! So happy to be part of this community.

path: 0 863365, hotness: undefined, score: 4, children: 0

sma3in 4 points 3 years ago

good vibes!! thank you for your work

path: 0 902694, hotness: undefined, score: 4, children: 0

mkhopper 4 points 3 years ago

Awesome news. Thanks for all the hard work.

path: 0 873713, hotness: undefined, score: 4, children: 0

comcreator 4 points 3 years ago

The instance seems to be much better. Posting and commenting is not taking as long and loading times are way better. I hope things can stay this good or even get better.

path: 0 963555, hotness: undefined, score: 4, children: 0

june 4 points 3 years ago

Was waiting for this update. Woke up a couple hours ago and Lemmy was working great so I knew y’all put in some time. Thanks for your efforts!

path: 0 850964, hotness: undefined, score: 4, children: 0

Labotomized 4 points 3 years ago

Thank you so much! I will be donating a few cappuccinos your way when my next check arrives. I really appreciate how awesome of a community you’ve brought together & all of the transparency with the updates (and the frequency) is astounding! Keep up the great work but don’t forget to take breaks :)

path: 0 864481, hotness: undefined, score: 4, children: 0

Pregnenolone 4 points 3 years ago

It’s very much helping the third party apps as well. Memmy is running way smoother now

path: 0 853716, hotness: undefined, score: 4, children: 0

HeyThisIsntTheYMCA 4 points 3 years ago

That's a very phallic bandwidth image Dr Freud

path: 0 854635, hotness: undefined, score: 4, children: 0

Xepher 4 points 3 years ago

Good job on troubleshooting!

Have you looked into possibly migrating to kubernetes or some other form of docker container management/orchestration system to help with automatic scaling and load balancing?

path: 0 850853, hotness: undefined, score: 4, children: 1

ruffsl 3 points 3 years ago

Don't quote me, but I recall reading on GitHub that there are a few things to be refactored before Lemmy can support horizontal scaling approaches.

path: 0 850853 858874, hotness: undefined, score: 3, children: 0

DV8 4 points 3 years ago

Please recommend people to update their app in a topic title. Connect couldn't even load a topic without failing out today. An update fixed it, but I had to manually force it because it didn't apply automatically.

This will drive people away. Literally none of the communities I subscribe to on world even seemed to have a new comment.

path: 0 866442, hotness: undefined, score: 4, children: 0

lazycouchpotato 4 points 3 years ago

Thank you to everyone that helped.

path: 0 862223, hotness: undefined, score: 4, children: 0

InverseParallax 4 points 3 years ago

That's some pretty heroic shit right there.

You just took lemmy from something I'm willing to live with in the short term in hope it gets better, to something I am fully satisfied with.

Now let's grow so we can fuck it up all over again!!!

path: 0 874438, hotness: undefined, score: 4, children: 0

akippnn 4 points 3 years ago

Thank you so much for the hard work, time and money you spend into making lemmy.world run very smoothly. This much transparency is awesome for something that's being used so massively.

path: 0 873600, hotness: undefined, score: 4, children: 0

jungle 3 points 3 years ago

path: 0 911538, hotness: undefined, score: 3, children: 1

ArrogantAnalyst 3 points 3 years ago

path: 0 903038, hotness: undefined, score: 3, children: 0

fubo 3 points 3 years ago

Woo hoo, nice graphs! Monitoring data that shows big improvements is always fun to see.

path: 0 863397, hotness: undefined, score: 3, children: 0

Sylvannes 3 points 3 years ago

Things are looking a lot more stable, thank you (all) for the amazing work!

path: 0 849697, hotness: undefined, score: 3, children: 0

kayaven 3 points 3 years ago

The changes are very much noticeable, thanks as always everyone!

path: 0 849704, hotness: undefined, score: 3, children: 0

bluemoose 3 points 3 years ago

Thanks for all the work you've put into this!

path: 0 864148, hotness: undefined, score: 3, children: 0

CthuluVoIP 3 points 3 years ago

Things feel a ton better today. Thank you for all the hard work!

path: 0 853757, hotness: undefined, score: 3, children: 0

p0ppe 3 points 3 years ago

Great work! Awesome to see how fast the technical side of the Lemmyverse is evolving and improving!

path: 0 850602, hotness: undefined, score: 3, children: 0

RipleyRiley 3 points 3 years ago

Upvotes appear to be working but I still can't post images. The post displays the infinite spinning loading circle.

path: 0 850988, hotness: undefined, score: 3, children: 4

ruud 10 points 3 years ago

How large is the image (both in dimensions and filesize) ? What image type is it? Is it in a post, comment?

path: 0 850988 851223, hotness: undefined, score: 10, children: 3

RipleyRiley 5 points 3 years ago

209KB, 720x891px, jpg. It was a post.

path: 0 850988 851223 851304, hotness: undefined, score: 5, children: 2

ruud 8 points 3 years ago

Do you see the separate line for uploading the image? If not, can you refresh the browser?

path: 0 850988 851223 851304 851462, hotness: undefined, score: 8, children: 1

RipleyRiley 3 points 3 years ago

I see the separate line for uploading images.

path: 0 850988 851223 851304 851462 851783, hotness: undefined, score: 3, children: 0

unreachable 3 points 3 years ago

am i amazed by you all guys work?!

i absolutely am

path: 0 880100, hotness: undefined, score: 3, children: 0

Aurix 3 points 3 years ago

The new updates made a ton of improvements. Thank you very much. It is near ideal now.

path: 0 866227, hotness: undefined, score: 3, children: 0

Sterben 3 points 3 years ago

Good job! 🤙

path: 0 866884, hotness: undefined, score: 3, children: 0

trouser_mouse 3 points 3 years ago

Awesome update, great to see and feel the progress from all the hard work!

path: 0 850862, hotness: undefined, score: 3, children: 0

ralino 3 points 3 years ago

Everything is much faster and smoother today. Thanks devs!

path: 0 854476, hotness: undefined, score: 3, children: 0

Landi 3 points 3 years ago

Thanks for your efforts and improvements. Keep it going ♥️🙏

path: 0 850942, hotness: undefined, score: 3, children: 0

Richard 3 points 3 years ago

Minor thing but over night both wefwef and Memmy clients are showing the wrong comment score (karma) against my profile, and given they are showing the same amount I assume it’s related to API fed data. Value was correct yesterday. Easy for me to confirm given I have only 2 dozen posts and the value has dropped to single digits.

Not a biggie, but figured I’d report it in case there was some issue causing that. Might be some optimisation around indexing or something has intentionally or unintentionally impacted that.

Otherwise the service feels much more stable currently. No timeouts today where it’s been very frequent the past few days. Nice job. 👍

path: 0 876160, hotness: undefined, score: 3, children: 2

ulu_mulu 3 points 3 years ago

Fantastic job in troubleshooting and submitting PRs! Improvements are very noticeable already :)

path: 0 851209, hotness: undefined, score: 3, children: 0

Yoz 3 points 3 years ago

LEGEND!

path: 0 867155, hotness: undefined, score: 3, children: 0

DuskLoaf 3 points 3 years ago

Thanks for the update! Things seem way speedier now ^^

path: 0 851271, hotness: undefined, score: 3, children: 0

Tygr 3 points 3 years ago

You all are champions today. When I loaded this up today, I was very happy and impressed. I knew tweaks were made.

path: 0 851427, hotness: undefined, score: 3, children: 0

TrueStoryBob 3 points 3 years ago

Running so much smoother! You guys rock!

path: 0 856554, hotness: undefined, score: 3, children: 0

wozomo 3 points 3 years ago

Thank you guys for all the hard work!!

path: 0 856431, hotness: undefined, score: 3, children: 0

Vlixz 3 points 3 years ago

Amezing improvement! Lemmy felt so much more responsive suddenly today, now it makes sense why :3

path: 0 854604, hotness: undefined, score: 3, children: 0

Sinister_Grape 3 points 3 years ago

Seemed to be running a lot smoother, thank you Ruud!

path: 0 853739, hotness: undefined, score: 3, children: 0

trambe 3 points 3 years ago

Can confirm, having way less problem browsing lemmy right now. Thank you admins!

path: 0 853248, hotness: undefined, score: 3, children: 0

malloc 3 points 3 years ago

damn bro, y’all coming in clutch to improve stability of this lemmy instance.

Good shit bros. Hope to contribute upstream and find more performance related bugs. I browsed the code for lemmy, and could not find any performance tests.

I can probably start there.

path: 0 863834, hotness: undefined, score: 3, children: 0

Anaralah_Belore223 2 points 3 years ago

path: 0 853635, hotness: undefined, score: 2, children: 0

CataclysmZA 2 points 3 years ago

Stuff is pretty smooth now!

path: 0 854759, hotness: undefined, score: 2, children: 0

GeneralStrike 2 points 3 years ago

Absolutely great job! Everything feels buttery smooth today, and I’m yet to see any errors.

path: 0 858636, hotness: undefined, score: 2, children: 0

dq9 2 points 3 years ago

Thank you so much for all your hard work!

path: 0 873742, hotness: undefined, score: 2, children: 0

Desistance 2 points 3 years ago

It's definitely snappier than before. Constantly having to refresh got old quick.

path: 0 853090, hotness: undefined, score: 2, children: 0

Zenbach 2 points 3 years ago

Working great thanks for all you do!

path: 0 858314, hotness: undefined, score: 2, children: 0

Beardliest 2 points 3 years ago

Really appreciate all the hard work going on behind the scenes! Feels night and day different after the changes. Also appreciate the transparency. Nice to see in this day and age.

path: 0 850179, hotness: undefined, score: 2, children: 0

Woshimaraci 2 points 3 years ago

Noice!

path: 0 865326, hotness: undefined, score: 2, children: 0

russjr08 2 points 3 years ago

Looks really good, and have definitely noticed less issues just over the last couple of hours. Great job folks!

path: 0 854803, hotness: undefined, score: 2, children: 0

Variden3301 2 points 3 years ago

Thank you for your work people. You are doing great.

path: 0 863329, hotness: undefined, score: 2, children: 0

decadentrebel 2 points 3 years ago

Awesome job as always, @ruud@lemmy.world

But these grey fonts on the sidebar will take some getting used to.

path: 0 852113, hotness: undefined, score: 2, children: 0

Safeguard77 2 points 3 years ago

Cheers!

path: 0 905010, hotness: undefined, score: 2, children: 0

patachu 2 points 3 years ago

Thank you to all the code monkeys for getting things straightened out!

System appears to be working as intended

path: 0 854898, hotness: undefined, score: 2, children: 0

CupDock 2 points 3 years ago

So much smoother! Thank you for all your charity and hard work!

path: 0 851747, hotness: undefined, score: 2, children: 0

Graphy 2 points 3 years ago

I read this post after browsing for an hour and I’ve definitely felt a change. Great work!

path: 0 863986, hotness: undefined, score: 2, children: 0

regex1883 2 points 3 years ago

Thank you!

path: 0 874287, hotness: undefined, score: 2, children: 0

Narann 2 points 3 years ago

I can't imagine the amount of work Lemmy's devs and ITs are under since few days, but those are important for the future of Lemmy. Keep the good work! You're awesome!

path: 0 853075, hotness: undefined, score: 2, children: 0

FlaxPicker 2 points 3 years ago

Not that it really matters, but did anyone else notice their comment/post points get deleted after this update? Seems odd.

path: 0 854048, hotness: undefined, score: 2, children: 4

ruud 4 points 3 years ago

What points?

path: 0 854048 856081, hotness: undefined, score: 4, children: 3

FlaxPicker 2 points 3 years ago

Its not on the default lemmy-ui on the web, but many third party apps for lemmy are showing total comment and post “scores” on user’s profile pages. Not exactly sure how they are calculating that, but ive noticed that those scores were wiped for at least my account today. Again, not a big deal, just thought it was strange how that would happen. It is showing like this in memmy as well as wefwef so it’s definitely some data out in the lemmy verse that went bad. it is starting to “count” stuff from comments ive made today though so, who knows.

path: 0 854048 856081 857648, hotness: undefined, score: 2, children: 2

ruud 3 points 3 years ago

Hmm that's odd. @aeharding@lemmy.world do you know why score counts are wiped? I have no clue how they are calculated.

path: 0 854048 856081 857648 860366, hotness: undefined, score: 3, children: 1

aeharding 3 points 3 years ago

It looks like on the person_view counts object API response coming back from Lemmy.world, the amounts for scores are not right. Hmmm

path: 0 854048 856081 857648 860366 860574, hotness: undefined, score: 3, children: 0

HybridSarcasm 2 points 3 years ago

Wow! Your commitment and diligence is admirable!

path: 0 851862, hotness: undefined, score: 2, children: 0

doryx 2 points 3 years ago

Well done!

path: 0 866394, hotness: undefined, score: 2, children: 0

boeman 2 points 3 years ago

Awesome work! I'm seeing a lot less errors and I can actively scroll through posts quickly now.

I'm in awe of how quickly these issues are being resolved.

Thank you for all the hard work.

path: 0 855649, hotness: undefined, score: 2, children: 0

wit 2 points 3 years ago

Are you guys able to create reports? I am not.. It keeps spinning.

path: 0 853238, hotness: undefined, score: 2, children: 0

burgundymyr 2 points 3 years ago

This is a lot of work, I genuinely feel honored by the service of everyone involved. Thank you!

path: 0 873025, hotness: undefined, score: 2, children: 0

cheeseblintzes 2 points 3 years ago

Thank you guys so much for all you do.

path: 0 865150, hotness: undefined, score: 2, children: 0

wanderingmagus 2 points 3 years ago

Thanks for all your hard work. The change is absolutely noticeable.

path: 0 851148, hotness: undefined, score: 2, children: 0

CrypticCoffee 2 points 3 years ago

That is so hot!

Thanks for sharing :).

Loving the transparency.

path: 0 851150, hotness: undefined, score: 2, children: 0

Slopz 2 points 3 years ago

Sweet!

path: 0 863499, hotness: undefined, score: 2, children: 0

devopspalmer 2 points 3 years ago

Amazing work @ruud@lemmy.world and admin team! Good to see some really awesome progress!

path: 0 865036, hotness: undefined, score: 2, children: 0

carroarmato0 2 points 3 years ago

Thanks for the tremendous work!

path: 0 865107, hotness: undefined, score: 2, children: 0

sugarfree 2 points 3 years ago

It's great to see such transparent and informative updates on this.

path: 0 863002, hotness: undefined, score: 2, children: 0

SteveJobs 2 points 3 years ago

Thanks for the hard work!

path: 0 849924, hotness: undefined, score: 2, children: 0

TheVampireSaga 1 point 3 years ago

Thank god, wasn't able to use the site lol

path: 0 873584, hotness: undefined, score: 1, children: 0

chairman 1 point 3 years ago

You guys are doing god's work. Thank you thank you thank you...

path: 0 873457, hotness: undefined, score: 1, children: 0

Rooki 1 point 3 years ago

The 502 error still seems to be common by me. Vut they are less permanent. Before it stayed multiple refreshes now it is safe to say after 1 reload it is most of the time gone.

path: 0 874623, hotness: undefined, score: 1, children: 0

Reverendender 1 point 3 years ago

I tried to enable push notifications and the app crashed. Also, how do I create a community?

path: 0 876207, hotness: undefined, score: 1, children: 0

karthan 1 point 3 years ago

Glorious, Ruud!

path: 0 904350, hotness: undefined, score: 1, children: 0

zikk_transport2 1 point 3 years ago

Can you describe your setup? Single instance or multi-node setup behind load balancer?

path: 0 926960, hotness: undefined, score: 1, children: 1

ebits21 1 point 3 years ago

Great work!

path: 0 850373, hotness: undefined, score: 1, children: 0

porksandwich9113 1 point 3 years ago

Sounds like we need to comment more to me. Great work all around!

path: 0 850536, hotness: undefined, score: 1, children: 0

IowaMan 1 point 3 years ago

Building history one update at a time 😎

path: 0 850362, hotness: undefined, score: 1, children: 0

mintiefresh 1 point 3 years ago

Awesome. Great work all!

path: 0 850648, hotness: undefined, score: 1, children: 0

Potato_in_my_anus 1 point 3 years ago

Loading is faster, thank you guys 👍

path: 0 850567, hotness: undefined, score: 1, children: 0

Tolstoy 1 point 3 years ago

Huge thank you for all your hard work^^

path: 0 857074, hotness: undefined, score: 1, children: 0

grumpyrico 1 point 3 years ago

path: 0 858883, hotness: undefined, score: 1, children: 0

ghariksforge 1 point 3 years ago

You guys are amazing!

path: 0 859890, hotness: undefined, score: 1, children: 0

AMillionMonkeys 1 point 3 years ago

Everything is feeling great so far. The only bug I'm encountering is that when opening a thread (in Firefox on desktop) it auto-scrolls down past the content to the replies.

path: 0 861277, hotness: undefined, score: 1, children: 0

daniskarma 1 point 3 years ago

Amazing guys. I did donate so you can keep up with things. I really want Lemmy to last.

path: 0 864442, hotness: undefined, score: 1, children: 0

AlmightySnoo 1 point 3 years ago

I think some of the empty page issues are back now?

path: 0 865774, hotness: undefined, score: 1, children: 0

Lanusensei87 1 point 3 years ago

Yay, I can finally log in on Chrome!

path: 0 871668, hotness: undefined, score: 1, children: 0

xaon_rider92 1 point 3 years ago

Thanks admins, things feel much smoother now. 👍

path: 0 872326, hotness: undefined, score: 1, children: 0

UniversalFlamingo 1 point 3 years ago

Amazing results! Thank you for all of your hard work. The collaboration among the admins is inspiring.

path: 0 872236, hotness: undefined, score: 1, children: 0

cheerjoy 1 point 3 years ago

I love how open you guys are when it comes to running the server! Much nicer than lemmy.ml imo

path: 0 873186, hotness: undefined, score: 1, children: 1

neuromancer 1 point 3 years ago

path: 0 859229, hotness: undefined, score: 1, children: 0

thetorrentio 1 point 7 months ago

path: 0 20689030, hotness: undefined, score: 1, children: 0

lemmyworld

@lemmy.world

login for more options

This Community is intended for posts about the Lemmy.world server by the admins.

Follow us for server news 🐘

Outages 🔥

https://status.lemmy.world

For support with issues at Lemmy.world, go to the Lemmy.world Support community.

Support e-mail

Any support requests are best sent to info@lemmy.world e-mail.

Report contact

DM https://lemmy.world/u/lwreport
Email report@lemmy.world (PGP Supported)

Donations 💗

If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.

If you can, please use / switch to Ko-Fi, it has the lowest fees for us

Join the team

go to feed...

lemmyworld

@lemmy.world

login for more options

This Community is intended for posts about the Lemmy.world server by the admins.

Follow us for server news 🐘

Outages 🔥

https://status.lemmy.world

For support with issues at Lemmy.world, go to the Lemmy.world Support community.

Support e-mail

Any support requests are best sent to info@lemmy.world e-mail.

Report contact

DM https://lemmy.world/u/lwreport
Email report@lemmy.world (PGP Supported)

Donations 💗

If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.

If you can, please use / switch to Ko-Fi, it has the lowest fees for us

Join the team

go to feed...