The Death of Decentralized Email

2 years ago by makeasnek to c/opensource

A historical review of the multi-decade centralization and capture of the email protocol.

Interesting history and analysis of SMTP's history. How can we prevent fedi and other open protocols from suffering the same fates?

observantTrapezium 89 points 2 years ago

You can't successfully use a home email server.

Mostly true (server can be home but using the ISP network directly probably won't work)

You can't successfully use an email server on a (cloud) VPS.

Bullshit

You can't successfully use an email server on a bare metal machine in your own datacenter.

Bullshit

As such, it is my distinct displeasure to declare the death of SMTP. The protocol is no longer usable. And as we can see, this devolution occurred organically.

Bullshit

path: 0 11152036, hotness: undefined, score: 89, children: 25
ramble81 33 points 2 years ago

You can’t successfully use an email server on a bare metal machine in your own Datacenter

Calling complete BS on that. I work in a medium size company and we do just that. Don’t know what he’s thinking.

path: 0 11152036 11154348, hotness: undefined, score: 33, children: 0
ikidd 10 points 2 years ago

I'm going to add "bullshit" to the first. I've gone 2 decades running a few email domains on my home servers, on 3 different ISPs. Its not rocket surgery.

path: 0 11152036 11162504, hotness: undefined, score: 10, children: 6
coronach 4 points 2 years ago

All the ISPs I've used block the relevant ports.

path: 0 11152036 11162504 11163098, hotness: undefined, score: 4, children: 0
i_am_hiding 3 points 2 years ago

I've been running one with a dozen or more users on bare metal at home for the last two years. A little bit of spam but otherwise fine. No deliverability issues or anything.

path: 0 11152036 11162504 11164887, hotness: undefined, score: 3, children: 0
JameUwU 0 points 2 years ago
path: 0 11152036 11162504 11177251, hotness: undefined, score: 0, children: 3
kalpol 1 point 2 years ago

Same here. Static IP though. I did set up another experiment with a haproxy vps just to see if I could do it if I lost my static IP, worked perfectly done that way too.

Fail2ban, pfblocker, and soamassassin work great.

path: 0 11152036 11162504 11177251 11276259, hotness: undefined, score: 1, children: 2
JameUwU 0 points 2 years ago
path: 0 11152036 11162504 11177251 11276259 11279724, hotness: undefined, score: 0, children: 1
xyguy 10 points 2 years ago

Can, yes.

Should, maybe.

Enjoy doing, unlikely.

And for sure your home isp has all the email ports blocked upstream.

With all that being said, to call SMTP dead is wildly insane. I do figure it will die someday though. Probably around the same time of universal IPV6 adoption during the year of the linux desktop.

path: 0 11152036 11155572, hotness: undefined, score: 10, children: 5
denshirenji 3 points 2 years ago

My ISP doesn't. It an electric company that offers fiber, so not your typical telecommunications company. Still though, not a single blocked port.

On topic, I tried an email server and it is too much of a pain in the ass IMHO, without the requisite training and experience, but certainly not impossible.

path: 0 11152036 11155572 11160249, hotness: undefined, score: 3, children: 4
dgriffith 3 points 2 years ago

My most recent ISP does CGNAT. They don't hide it, it's mentioned in their support pages. A quick email is all it takes to switch you over to an open address though.

Anyway I've got a $5/mo server with akami that looks after my email and it's associated domain.

It took about three hours of following a guide to set up DMARC and etc etc and it works unobtrusively, and is about ten times faster than my old ISP IMAP account that I had for about twenty years.

path: 0 11152036 11155572 11160249 11169435, hotness: undefined, score: 3, children: 3
denshirenji 3 points 2 years ago

Do you have a link to the guide by any chance? I might try it again using one of my throw away domains as a test.

path: 0 11152036 11155572 11160249 11169435 11173469, hotness: undefined, score: 3, children: 2
smiletolerantly 3 points 2 years ago

Yeah. I've had zero problems hosting my mail on a bare metalachine in a datacenter. They arrive just like they should, plus it's just so freeing to host it yourself.

path: 0 11152036 11157017, hotness: undefined, score: 3, children: 0
makeasnek 0 points 2 years ago

Sure, you can run one, good luck getting even a halfway decent delivery rate to mailboxes at any major mail provider. Even if they never receive a spam message from your server, your server is an "unknown" which counts against you. And if one person in your small company of 10 or 100 or even 1000 people gets their e-mail hacked and sends spam? Prepare for the rest of them to get punished for it. Running an SMTP server is a nightmare which is why, over time, more and more of the economy has just shifted their SMTP servers to organizations who professionally run SMTP servers instead of having their own.

path: 0 11152036 11152095, hotness: undefined, score: 0, children: 9
the_crotch 32 points 2 years ago

Set up dkim/SPF properly, make sure the ip you plan to use is clean before you start, sign up for MXtoolbox blacklist alerts and if you get on a blacklist (doesn't happen often if you do a bare minimum of proactive security), you request removal. It's really not hard.

path: 0 11152036 11152095 11152587, hotness: undefined, score: 32, children: 0
bahuma20 25 points 2 years ago

I am running my own mailsever for over 10 years without any blacklisting problems...

path: 0 11152036 11152095 11152444, hotness: undefined, score: 25, children: 3
Corgana 9 points 2 years ago

Right, but try doing that with a 10 day old server created in 2024. That's the hurdle people are referring to.

path: 0 11152036 11152095 11152444 11153194, hotness: undefined, score: 9, children: 2
Samsy 11 points 2 years ago

This works, too. It's actually common that your dmarc-entry needs some time to be accepted everywhere. Wait a few days more and your mails don't hit the spam folder on google and outlook.

path: 0 11152036 11152095 11152444 11153194 11154780, hotness: undefined, score: 11, children: 0
ShortN0te 9 points 2 years ago

Just did that a few months ago. No issues yet.

path: 0 11152036 11152095 11152444 11153194 11155968, hotness: undefined, score: 9, children: 0
majestictechie 19 points 2 years ago

I work as a Sysadmin for a web host who sells VPS's. I've helped many people setup domains on their server to cover SPF, DKIM and DMARC passes on a daily basis. Most use these for personal or business level mail delivery without issue.

Are there hurdles to overcome? Sure. But it's not exactly hard as long as you have a IP that's isnt a poor reputation (which as an ISP we help delist and improve). But it's not impossible.

Its more "convenient" to use a third party mail provider just as Office365 since you pass on all that setup and responsibility onto their framework, but it's not hard to setup a decent level of mail service yourself.

path: 0 11152036 11152095 11152896, hotness: undefined, score: 19, children: 0
ErwinLottemann 9 points 2 years ago

have mine running for nearly 20 years now and never had any major issues with delivery 🤷

path: 0 11152036 11152095 11152369, hotness: undefined, score: 9, children: 0
Supermariofan67 6 points 2 years ago

I've never had any issues getting mail delivered to major providers

path: 0 11152036 11152095 11153762, hotness: undefined, score: 6, children: 0
digdilem 0 points 2 years ago

You're spot on, and even smaller ISPs routinely get blocked by larger hosters (anyone who doubts this, please look around for the many stories along the lines of "gmail silently drops my email")

Residential IP blocks are scored much higher and given a negative trust from the start - not surprising since that's where much of the world's spam comes from through compromised computers, routers etc.

path: 0 11152036 11152095 11152623, hotness: undefined, score: 0, children: 0
halm 70 points 2 years ago

I know there are problems with big email providers subverting decentralisation to benefit their business models, and throttling mail from independent or self-hosted domains. But I couldn't take the analysis seriously past this statement:

You may know me as a Bitcoin educator and engineer.

Yeah well, in that case, fuck you and the hypercapitalist horse you rode in on.

path: 0 11156335, hotness: undefined, score: 70, children: 22
refalo 4 points 2 years ago

"Sent from my iPhone"

path: 0 11156335 11169934, hotness: undefined, score: 4, children: 2
halm 3 points 2 years ago

It's an ungoogled Android actually, but I can see how that ruins your joke 🙂

path: 0 11156335 11169934 11171768, hotness: undefined, score: 3, children: 1
JameUwU 4 points 2 years ago
path: 0 11156335 11169934 11171768 11177181, hotness: undefined, score: 4, children: 0
sibachian 0 points 2 years ago

well to be fair bitcoin is on point as a decentralized currency.

path: 0 11156335 11182088, hotness: undefined, score: 0, children: 0
makeasnek -6 points 2 years ago

You may know me as a Bitcoin educator and engineer.

Yeah well, in that case, fuck you and the hypercapitalist horse you rode in on.

This guy is a protocol engineer, talking about protocols. You may not like like Bitcoin, but it's pretty hard to argue it's not one of the most successful, widely-used, and forked open source protocols developed in the last several decades. Bitcoin core is in the top 100 starred repos on Github. It has a unicode character.

Bitcoin's market cap (> 1 trillion USD) is bigger than Sweden's GDP and it moves billions of dollars around the world every year. You can use it to send money to anybody with a phone and a halfway reliable internet connection in under a second for pennies in fees, and it settles instantly. And it's been working for 15 years without a single hour of downtime, bank holiday, or hack despite pandemics, wars, financial crises, and attempted bans by global powers.

Like, be mad if you want, but it's a pretty successful and robust protocol. And if you don't like it, you can fork it and change it, because it's open source.

path: 0 11156335 11158351, hotness: undefined, score: -6, children: 7
vrighter 18 points 2 years ago

http, https, ssh, ntp, ftp. These are all algorithms some of us use every day. Bitcoin is a protocol, true, but it's not a good one. And it's one that most people have not used, and don't intend to

It has a lot of forks? that is neither here nor there. it's a tech buzzword. of course there are going to be a lot of forks. Do any of them actually go anywhere though? not really

path: 0 11156335 11158351 11161436, hotness: undefined, score: 18, children: 0
Sethayy 6 points 2 years ago

God damn bitcoin >> SSH?? (1999)

That's a pretty steamy take if I've ever heard one

path: 0 11156335 11158351 11160843, hotness: undefined, score: 6, children: 4
chebra 2 points 2 years ago path: 0 11156335 11158351 11160843 11167597, hotness: undefined, score: 2, children: 3
Sethayy 2 points 2 years ago

'One of the most successful, widely used, and forked protocols'

SSH is also a protocol, tho I'll admit maybe its 'one of the few' above bitcoin - but I can come up with a page of examples that top it if you need (HTTP, TCP, UDP, RTSP, RGMII...)

path: 0 11156335 11158351 11160843 11167597 11167598, hotness: undefined, score: 2, children: 2
halm 3 points 2 years ago

No, see — if I dont like it I don't need to fork it. I can just leave it and all its forks the hell alone. I'd do the same for national currencies if I could, cryptocurrencies are just the same bullshit without the regulatory checks and balances.

TL;DR — I see what you're selling and I'm not buying it.

path: 0 11156335 11158351 11164819, hotness: undefined, score: 3, children: 0
___ -25 points 2 years ago

Bitcoin is hypercapitalist? A decentralized value store not controlled by any one country and immune to money printing inflation? What are you smoking?

path: 0 11156335 11158561, hotness: undefined, score: -25, children: 9
kogasa 17 points 2 years ago

Bitcoin is more widely seen as a vehicle for speculation rather than a decentralized currency. Unlucky.

path: 0 11156335 11158561 11164954, hotness: undefined, score: 17, children: 1
sibachian 1 point 2 years ago

just like email.

path: 0 11156335 11158561 11164954 11182095, hotness: undefined, score: 1, children: 0
Sethayy 12 points 2 years ago

Capitalists will even sell you communism if it makes them a dime, end result is cryptocurrency is half assed solving a problem that doesn't really exist.

Like inflation is a great example, you shouldnt have to add modifiers onto its definition, inflation is inflation - bitcoin by design must inflate

path: 0 11156335 11158561 11160782, hotness: undefined, score: 12, children: 3
___ -5 points 2 years ago

By design, it will slowly stop inflating at the snails pace it does vs unpegged paper currency.

A central bank regulating where money is printed and to whom it’s distributed at what contrived rates is horrifying. It’s also the default in most of the world.

You can trade bitcoin and use it as a currency in a non-capitalist market. The fact that it has been abused and traded into stratospheric value is a result of manipulation, sanctioned exchanges, and propaganda.

Bitcoin just allows you to write debits and credits on a distributed, verified, ledger. That’s it really. How the market is regulated is on the people, not the technology. There is nothing inherently capitalist about the technology other than allowing any individual to trade value with another in a free market manner. You would be trying to escape supply and demand dynamics to remove that “capitalist” aspect of it.

The power draw on the other hand.. the first imagining of a digital decentralized and distributed currency was bound to have some problems.

path: 0 11156335 11158561 11160782 11164383, hotness: undefined, score: -5, children: 2
Sethayy 1 point 2 years ago

Snails pace??

You wanna take a 2 second peek at the value of bitcoin over the past decade (over 7 years bitcoin is about 3000%, USD is approximately 32% over 10)

Banks do suck but there's absolutely no reason rich people can't manipulate bitcoin - maybe even easier than traditional money.

Put your money in gold or something that actually exists instead of an imaginary number that's limited in supply

path: 0 11156335 11158561 11160782 11164383 11167704, hotness: undefined, score: 1, children: 1
Darohan 11 points 2 years ago

I mean, it's been shown that it's relatively easy for a big company to control the price of Bitcoin, and there's nothing more capitalist than wanting to get away from the control of countries and states that might get in the way of making as much profit as possible so,,,, yeah no I'd say hypercapitalist is a valid accusation. Bitcoin was designed to beat the big banks and capitalist status-quo, but I don't think that we can pretend it succeeded anymore.

path: 0 11156335 11158561 11159539, hotness: undefined, score: 11, children: 2
Sethayy 3 points 2 years ago

Its like China calling itself communist, like sure maybe its 0.1% more than ultra capitalist US, but both they're both shit

path: 0 11156335 11158561 11159539 11160803, hotness: undefined, score: 3, children: 0
___ -4 points 2 years ago

No tech is perfect. And the current bitcoin is not the same as the original client. It has been modified to allow for abuse and control. The fact that we allow this to take place is more a reflection of our governments aiming to control it than any inherent property of the currency.

Big banks would have far less control if you couldn’t print sanctioned currency to buy as much bitcoin as they want to play with the value set by sanctioned exchanges.

I agree that bitcoin is capitalist like most monetary bills in a free market. I disagree it’s more capitalist than what we have now. It’s just being propagandized and veiled from the underlying technology to make it seem so.

path: 0 11156335 11158561 11159539 11164444, hotness: undefined, score: -4, children: 0
benjhm 48 points 2 years ago

I don't buy this. I'm still using SMTP on my own domain and it’s working fine, a bit of spam but not unmanageable, real messages get read. Main challenge is digesting so many potentially-interesting list messages, indicating email's continued dominance for professional topics. Seems this author has another agenda.
Having said that, it's a pity the world never agreed a protocol for micro-payment for emails (and for many other services), which would resolve the spam problem, and not be a burden for honest users.

path: 0 11148871, hotness: undefined, score: 48, children: 1
angel 4 points 2 years ago

I also host my own mailserver and I agree that it mostly works fine. However, there are some email providers that cause trouble:

Google seems to randomly sort some of my mails into the recipients spam folder, while others are delivered fine to the respective inbox. It kinda sucks that you can never be sure whether the recipient actually received your mail or whether they just don't reply. My IP and domain are not blacklisted on any spam list; SPF, DKIM and DMARC are set up correctly as well.

Even worse is the Telekom (German ISP), who use an explicit whitelist of IP addresses (only IPv4 of course) and require you to display your contact information publicly on a website reachable via the same domain your mailserver uses. Once you've set this up you need to message them to be put on their whitelist. If you're not on their whitelist, they simply reject your mails, they are not even delivered to the spam folder (maybe it's not worse than Google, because you at least get a notice from your mailserver that your mail couldn't be delivered). In the end I decided that I don't care enough to comply with their regulations and just don't send any mails to Telekom customers.

Aside Google and Telekom, I've really never had any issues though.

path: 0 11148871 11153265, hotness: undefined, score: 4, children: 0
notannpc 28 points 2 years ago

Immediately skeptical by the ai generated tombstone as the article image, and the skepticism was warranted. Massive L take from a “bitcoin educator”.

path: 0 11157292, hotness: undefined, score: 28, children: 1
allywilson 4 points 2 years ago

Same, seen the AI generation and was out.

path: 0 11157292 11173115, hotness: undefined, score: 4, children: 0
amanda 24 points 2 years ago

I should have expected the rug-pull at the end when I read:

You may know me as a Bitcoin educator and engineer

However, I was still surprised!

path: 0 11149676, hotness: undefined, score: 24, children: 0
SorteKanin 16 points 2 years ago

Defederating bad actors/spammers should in theory be good enough? Domains aren't free and I don't think it's worth it for them to buy a new domain to just be able to spam for a short time again.

path: 0 11149260, hotness: undefined, score: 16, children: 24
makeasnek 27 points 2 years ago

Domains aren’t free and I don’t think it’s worth it for them to buy a new domain to just be able to spam for a short time again.

Literally what e-mail spammers do.

Agreed defederating can help solve obviously malicious instances, it doesn't solve spammers abusing good instances. E-mail and AP are very similar at a protocol structure level.

path: 0 11149260 11149445, hotness: undefined, score: 27, children: 11
SorteKanin 12 points 2 years ago

Is it though? Don't email spammers just spoof the domain or send without a domain? I'm not entirely sure if that's different from how the fediverse works. I'm not too knowledgeable about this topic.

path: 0 11149260 11149445 11149491, hotness: undefined, score: 12, children: 10
halm 9 points 2 years ago

Don’t email spammers just spoof the domain or send without a domain?

Very much so. Out of the spam that I do see in my inbox, the sender domains are usually spoofed, while the reply-to addresses are usually gmail.com, hotmail.com or outlook.com.

path: 0 11149260 11149445 11149491 11149711, hotness: undefined, score: 9, children: 0
the_crotch 5 points 2 years ago

You need to set up dkim to prevent spoofing. Each message sent has a digital signature that matches one on a DNS record for your domain. You can also set an SPF record, which will tell the recipient what up addresses are authorized to send mail on behalf of your domain.

The recipent must have policies in place that reject mail which fails dkim/spf

path: 0 11149260 11149445 11149491 11152647, hotness: undefined, score: 5, children: 0
makeasnek 4 points 2 years ago

Don’t email spammers just spoof the domain or send without a domain?

They do both, depending on the spammer and the type of spam they send. In e-mail, you have an e-mail server, you can use it to send mail to users on other e-mail servers. Each e-mail server can choose to accept or reject email from other e-mail servers based on whatever reason they want. AP/Lemmy/Mastodon is basically identical to this. I'm not sure how exactly bluesky is setup but I get the impression it's similar. In Nostr, servers aren't federated (each relay is seperate, if you want to send/recieve content to another user on a different relays you just talk to that relay directly instead of having "your relay" act as an intermediary), but the structure is still pretty similar.

Nostr does have this hashcash type system (requiring proof-of-work to weed out spam), but I haven't come across any relays that actually enforce it, it will be interesting to see if that changes in time. I also saw a GitHub issue about adding something similar to AP but I think they chose not to implement it.

path: 0 11149260 11149445 11149491 11149535, hotness: undefined, score: 4, children: 7
SorteKanin 7 points 2 years ago

Replying to your edit:

it doesn’t solve spammers abusing good instances

This is an instance moderation problem. If you're letting spammers in, you need to use a better application process or something similar to that. A big problem with email spam is that most email services allow anyone to sign up for free without any checks.

Ultimately defederating bad actors and defederating "good" actors who fail to moderate their own users is necessary.

path: 0 11149260 11149445 11149491 11149535 11149633, hotness: undefined, score: 7, children: 6
GolfNovemberUniform 4 points 2 years ago

But most people don't pay for software, especially if there are "free" and legal alternatives.

path: 0 11149260 11149364, hotness: undefined, score: 4, children: 11
SorteKanin 7 points 2 years ago

I'm not sure what you mean with that or how it relates to what I said, could you elaborate?

path: 0 11149260 11149364 11149507, hotness: undefined, score: 7, children: 10
GolfNovemberUniform 2 points 2 years ago

As I understood you said that we should make email paid to stop spammers.

path: 0 11149260 11149364 11149507 11149832, hotness: undefined, score: 2, children: 9
SorteKanin 6 points 2 years ago

Nono, I'm saying it costs to spam because spammers have to keep buying new domains as their previous domains get blocked or defederated.

path: 0 11149260 11149364 11149507 11149832 11149881, hotness: undefined, score: 6, children: 8
pyre 15 points 2 years ago

i can't read anything that's presented with that shitty cover image without a hint of irony

path: 0 11151099, hotness: undefined, score: 15, children: 0
digdilem 13 points 2 years ago

(This is as much an answer to some of the comments already raised, as to the article - which like most such personal pieces has pros and cons.)

As part of a previous job I used to host email for a small business - this was about 15 years ago. I ended up spending several hours to a day a week working on it; apologising to users, tracing and diagnosing missing sent email and the endless, ENDLESS arms war against incoming spam (phishing was much less of a problem then). The trust from the company in our email operation was very poor and you'd regularly hear someone apologising to a customer because we hadn't contacted them, or answered their email. The truth is much was going astray and staff were relying more on the phone than email because they knew it worked. You might guess from this that I'm terrible at running an email system but I don't think I am. I started moving email back in the late 80s when Fidonet was the thing, so I have some miles travelled. Tools have improved a bit since then, but so have those used by the bad guys.

I still consider one of the best things I did for that company was move our company email onto Gmail Business (which was free for us as a charity) Every single one of those problems went away immediately and suddenly I had a lot more time to do more important stuff. I would never self-host email again despite running several personal servers.

Plenty of people say they self-host just fine, and great for you if that's so. But the truth is you won't always know if your outbound mail silently gets dropped and you have a far higher chance of it arriving if it comes from a reputable source. There are a huge number of variables outside of your control. (ISP, your country, your region, your software, even the latency of your MX or DKIM responses factor into your reputation)

You take the decision on whether any perceieved risks of privacy through using a third party outweighs the deliverability and filtering issues of self hosting, but please don't say it's simple or reliable for everyone. If it's simple for you, you're either incredibly lucky or just not appreciating the problem.

path: 0 11152872, hotness: undefined, score: 13, children: 0
umami_wasbi 7 points 2 years ago

I never run a mail server but Google already placing my mail sent via my xyz domain hosted on proton to spam folder silently.

I guess running my own will be a lot worst.

P.S. I know that's a bad TLD choice, and I'm planning to migrate, but that will take a lots of time and work to the point I wonders if that worth it as I don't sent many anyways.

path: 0 11155797, hotness: undefined, score: 7, children: 2
ikidd 1 point 2 years ago

Don't even bother with an xyz domain. Pay for something useful, its not expensive via Namecheap or Porkbun

path: 0 11155797 11162699, hotness: undefined, score: 1, children: 1
umami_wasbi 1 point 2 years ago

I bought it about a year after general availability when I'm still in college, and doesn't know it will be this bad.

path: 0 11155797 11162699 11168750, hotness: undefined, score: 1, children: 0
ssm 5 points 2 years ago

And threads will be the death of decentralized lemmy. But we still have mailing lists, and most of my mails go to decentralized users on those lists. You just gotta know where to look, and you'll find gold.

path: 0 11174020, hotness: undefined, score: 5, children: 0
opensource
opensource

@lemmy.ml

login for more options
49559
3011
4288

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

Rules

  • Posts must be relevant to the open source ideology
  • No NSFW content
  • No hate speech, bigotry, etc

Related Communities

Community icon from opensource.org, but we are not affiliated with them.

go to feed...