How to harden against SSH brute-forcing?

2 years ago by someacnt to c/selfhosted

Recently, I discovered that SSH of my VPS server is constantly battered as follows.

Apr 06 11:15:14 abastro-personal-arm sshd[102702]: Unable to negotiate with 218.92.0.201 port 53768: no matching key exchange method found. Their offer: diffie>
Apr 06 11:30:29 abastro-personal-arm sshd[102786]: Unable to negotiate with 218.92.0.207 port 18464: no matching key exchange method found. Their offer: diffie>
Apr 06 11:45:36 abastro-personal-arm sshd[102881]: Unable to negotiate with 218.92.0.209 port 59634: no matching key exchange method found. Their offer: diffie>
Apr 06 12:01:02 abastro-personal-arm sshd[103019]: Unable to negotiate with 218.92.0.203 port 16976: no matching key exchange method found. Their offer: diffie>
Apr 06 12:05:49 abastro-personal-arm sshd[103066]: Unable to negotiate with 218.92.0.212 port 49130: no matching key exchange method found. Their offer: diffie>
Apr 06 12:07:09 abastro-personal-arm sshd[103077]: Connection closed by 162.142.125.122 port 56110 [preauth]
Apr 06 12:12:18 abastro-personal-arm sshd[103154]: Connection closed by 45.79.181.223 port 22064 [preauth]
Apr 06 12:12:19 abastro-personal-arm sshd[103156]: Connection closed by 45.79.181.223 port 22078 [preauth]
Apr 06 12:12:20 abastro-personal-arm sshd[103158]: Connection closed by 45.79.181.223 port 22112 [preauth]
Apr 06 12:21:26 abastro-personal-arm sshd[103253]: Connection closed by 118.25.174.89 port 36334 [preauth]
Apr 06 12:23:39 abastro-personal-arm sshd[103282]: Unable to negotiate with 218.92.0.252 port 59622: no matching key exchange method found. Their offer: diffie>
Apr 06 12:26:38 abastro-personal-arm sshd[103312]: Connection closed by 92.118.39.73 port 44400
Apr 06 12:32:22 abastro-personal-arm sshd[103373]: Unable to negotiate with 218.92.0.203 port 57092: no matching key exchange method found. Their offer: diffie>
Apr 06 12:49:48 abastro-personal-arm sshd[103556]: error: maximum authentication attempts exceeded for root from 98.22.89.155 port 53675 ssh2 [preauth]
Apr 06 12:49:48 abastro-personal-arm sshd[103556]: Disconnecting authenticating user root 98.22.89.155 port 53675: Too many authentication failures [preauth]
Apr 06 12:49:51 abastro-personal-arm sshd[103558]: error: maximum authentication attempts exceeded for root from 98.22.89.155 port 53775 ssh2 [preauth]
Apr 06 12:49:51 abastro-personal-arm sshd[103558]: Disconnecting authenticating user root 98.22.89.155 port 53775: Too many authentication failures [preauth]
Apr 06 12:49:53 abastro-personal-arm sshd[103561]: error: maximum authentication attempts exceeded for root from 98.22.89.155 port 53829 ssh2 [preauth]
Apr 06 12:49:53 abastro-personal-arm sshd[103561]: Disconnecting authenticating user root 98.22.89.155 port 53829: Too many authentication failures [preauth]
Apr 06 12:49:54 abastro-personal-arm sshd[103563]: Connection closed by 98.22.89.155 port 53862 [preauth]
Apr 06 12:50:41 abastro-personal-arm sshd[103576]: Invalid user  from 75.12.134.50 port 36312
Apr 06 12:54:26 abastro-personal-arm sshd[103621]: Connection closed by 165.140.237.71 port 54236
Apr 06 13:01:26 abastro-personal-arm sshd[103702]: Connection closed by 193.32.162.132 port 33380
Apr 06 13:03:40 abastro-personal-arm sshd[103724]: Unable to negotiate with 218.92.0.204 port 60446: no matching key exchange method found. Their offer: diffie>
Apr 06 13:11:49 abastro-personal-arm sshd[103815]: Received disconnect from 165.140.237.71 port 50952:11:  [preauth]
Apr 06 13:11:49 abastro-personal-arm sshd[103815]: Disconnected from authenticating user root 165.140.237.71 port 50952 [preauth]
Apr 06 13:19:08 abastro-personal-arm sshd[103897]: Unable to negotiate with 218.92.0.208 port 59274: no matching key exchange method found. Their offer: diffie>
Apr 06 13:33:36 abastro-personal-arm sshd[104066]: Received disconnect from 165.140.237.71 port 50738:11:  [preauth]
Apr 06 13:33:36 abastro-personal-arm sshd[104066]: Disconnected from authenticating user ubuntu 165.140.237.71 port 50738 [preauth]
Apr 06 13:34:50 abastro-personal-arm sshd[104079]: Unable to negotiate with 218.92.0.204 port 44816: no matching key exchange method found. Their offer: diffie>
Apr 06 13:50:32 abastro-personal-arm sshd[104249]: Unable to negotiate with 218.92.0.206 port 27286: no matching key exchange method found. Their offer: diffie>
Apr 06 13:51:58 abastro-personal-arm sshd[104261]: Received disconnect from 165.140.237.71 port 50528:11:  [preauth]
Apr 06 13:51:58 abastro-personal-arm sshd[104261]: Disconnected from authenticating user root 165.140.237.71 port 50528 [preauth]
Apr 06 14:01:25 abastro-personal-arm sshd[104351]: Invalid user  from 65.49.1.29 port 18519
Apr 06 14:01:28 abastro-personal-arm sshd[104351]: Connection closed by invalid user  65.49.1.29 port 18519 [preauth]

As you can see, it is happening quite frequently, and I am worried one might break in at some point. Since SSH access guards users with root-access, it can be quite serious once penetrated. How do I harden against these kind of attacks? Because this is VPS, disabling SSH is a no-go (SSH is my only entry of access). Are there ways to stop some of these attackers?

As always, thanks in advance!

CondorWonder 71 points 2 years ago

We can’t ever stop this kind of stuff, but with something like fail2ban you can set it up to block on too many failures.

Really though - ensuring your system is kept up to date and uses strong passwords or use a SSH keys is the best defence. Blocking doesn’t prevent them from trying a few times. Moving SSH to a non standard port will stop most of the automated attacks but it won’t stop someone who is dedicated.

path: 0 16274968, hotness: undefined, score: 71, children: 7
30p87 26 points 2 years ago

Move SSH to non-standard port, make endlessh use the default port. Only use SSH keys. Only allow correct users (so eg. your user and git/forgejo). Use fail2ban to aggressively ban (redirect to default port, so 22) and report to abuseipdb everything that fails to authenticate first try (wrong user, password instead of key), has non-compatible ciphers (generally, only allow TLS1.3 etc.), or fails in any other way. Just be sure that if you accidentally get banned yourself (eg. Ctrl+C-ing during authentication), you can use another IP (eg. force v4) for connecting.

path: 0 16274968 16276802, hotness: undefined, score: 26, children: 4
cron 9 points 2 years ago

Nice list of suggestions, but implementing all of them feels a little over-the-top.

path: 0 16274968 16276802 16278545, hotness: undefined, score: 9, children: 1
30p87 9 points 2 years ago

Tbh, I myself still have SSH on port 22. Firstly, because I'm lazy, and secondly ... yeah that's it. I'm honestly just lazy. But spam bots trying office/cookie123 are not a real threat, and anyone trying to actually target me will either have somehow acquired my key + password, use one of the probably many security issues that exist in the dozen services I selfhost, social engineer me into doing something (not saying I've given out my (old) KeePass password once, but it could be, as love makes blind (I still love her)), or just smash my kneecaps until I give out everything.

path: 0 16274968 16276802 16278545 16282015, hotness: undefined, score: 9, children: 0
bigBananas 2 points 2 years ago

Also, add 2FA

path: 0 16274968 16276802 16281766, hotness: undefined, score: 2, children: 1
30p87 1 point 2 years ago

But remember, on a third device. Not the one where your KeePass DB is one fingerprint away, and your private SSH key too.

path: 0 16274968 16276802 16281766 16282057, hotness: undefined, score: 1, children: 0
someacnt 1 point 2 years ago

Thanks, I will try fail2ban. I am using ED25519 for ssh keys, it seems like it's the best defense on the ssh side. Do you happen to know why this kind of attack is so prevalent?

path: 0 16274968 16284244, hotness: undefined, score: 1, children: 1
WhyJiffie 3 points 2 years ago

I'm not them, but among other reasons they are looking to build botnets (cryptomining, dosing, mass crawling), and they are searching for hosts with low security (or if you just made a mistake)

path: 0 16274968 16284244 16286480, hotness: undefined, score: 3, children: 0
Xanza 59 points 2 years ago
  1. Disable passwordless login.
  2. Disable password login.
  3. Require SSH keys
  4. Move SSH port to non-standard port
  5. Reject connections to port 22
  6. Install and enable fail2ban

About the best you can do.

path: 0 16279972, hotness: undefined, score: 59, children: 17
semperverus 12 points 2 years ago

Don't reject connections to port 22, honeypot it and ban on connection attempt.

path: 0 16279972 16282755, hotness: undefined, score: 12, children: 5
downhomechunk 5 points 2 years ago

I'd get myself banned this way. I forget the -p flag at least once per week.

path: 0 16279972 16282755 16301756, hotness: undefined, score: 5, children: 0
sugar_in_your_tea 3 points 2 years ago

honeypot

That's a lot more work.

path: 0 16279972 16282755 16283055, hotness: undefined, score: 3, children: 3
ILaughBecauseFunny 2 points 2 years ago

By all means, I am no expert, but isn't it "just" the fail2ban?

path: 0 16279972 16282755 16283055 16292199, hotness: undefined, score: 2, children: 2
sugar_in_your_tea 5 points 2 years ago

Fail2ban blocks IPs that fail to connect repeatedly. A honeypot pretends fails worked and gives them a worthless environment to try to exploit. The purpose of fail2ban is to block attacks, the purpose of a honeypot is amusement and to waste attackers' time.

path: 0 16279972 16282755 16283055 16292199 16295554, hotness: undefined, score: 5, children: 1
markstos 7 points 2 years ago

Using a nonstandard port doesn’t get you much, especially popular nonstandard ports like 2222.

I used that port once and just as much junk traffic and ultimately regretted bothering.

path: 0 16279972 16280093, hotness: undefined, score: 7, children: 10
friend_of_satan 20 points 2 years ago

My experience running several ssh servers on uncommon nonstandard ports for over 10 years has been that it has eliminated all ssh brute forcing. I don't even bother with fail2ban. I probably should though, just in case.

Also, PSA: if you use fail2ban, don't try tab completing rsync commands without using controlmaster or you will lock yourself out.

path: 0 16279972 16280093 16280345, hotness: undefined, score: 20, children: 0
irmadlad 5 points 2 years ago

My two cents: Using a nonstandard ssh port is good for dumping bots. True, you can easily do a port scan against a server and easily find all open ports nbd. But most off-the-shelf bots are looking for standard ports to penetrate. I know that when I format and reinstall the test server, as soon as I change the ssh port, bot noise goes down significantly. So, for a simple config edit and about 2 minutes of time, it seems worth the effort. It's just one layer tho. And yes, it goes without saying to pick a port other than 22, 222, 2222, etc.

path: 0 16279972 16280093 16280791, hotness: undefined, score: 5, children: 2
loudWaterEnjoyer 2 points 2 years ago

How about 22222?

path: 0 16279972 16280093 16280791 16298094, hotness: undefined, score: 2, children: 1
sugar_in_your_tea 3 points 2 years ago

Oh, that one's fine. Everyone knows that 5-digit ports add extra security, which is why WireGuard runs on port 51820 by default. You can verify this by checking the max port number, which is also 5-digits, computers just aren't powerful enough to crack 6-digit port numbers, so Linux hasn't bothered increasing it. /s

path: 0 16279972 16280093 16280791 16298094 16299075, hotness: undefined, score: 3, children: 0
cmnybo 4 points 2 years ago

It gets rid of most of the login attempts for me. I don't use a popular port though. Pick a 5 digit port so they have to put in some effort to find it.

path: 0 16279972 16280093 16280225, hotness: undefined, score: 4, children: 0
Xanza -1 points 2 years ago

Using a nonstandard port doesn’t get you much

Uhh... It gets you a lot. Specifically, unless you know the port you can't connect... So hey, there's that..

This community really says shit sometimes that makes me go cross-eyed....

path: 0 16279972 16280093 16361757, hotness: undefined, score: -1, children: 4
markstos 0 points 2 years ago

The top-rated answer to this question on the Security StackExhange is “not really”. https://security.stackexchange.com/...

On Serverfault, the top answer is that random SSH ports provide “no serious defense” https://serverfault.com/...

Or the answer here, highlighting that scanners check a whole range ports and all the pitfalls of changing the port. Concluding: “Often times it is simply easier to just configure your firewall to only allow access to 22 from specific hosts, as opposed to the whole Internet.” https://security.stackexchange.com/...

path: 0 16279972 16280093 16361757 16369086, hotness: undefined, score: 0, children: 3
Xanza 0 points 2 years ago

And I'm a CEHv7. A literal security professional--and I say that an overwhelming vast majority of attacks against servers using SSH are going to come over the default port. Quite literally 99%. This means that you can lower your attack surface by exactly 99% by simply changing the default SSH port...

Those posts provide no meaningful insight and what they say is by the very technical of all interpretations is correct, I absolutely disagree with these statements. What they mean to say is that simply changing the default SSH port isn't alone I means of strictly protecting yourself. Meaning you shouldn't change the default SSH port and think that your server is secured because it's not.

Quite the different interpretation than me saying it should be mandatorily a part of your security strategy.

In protecting yourself against port scanning is trivial.

Anyone underestimating the power of changing The default SSH port is someone who's opinion I can safely disregard.

path: 0 16279972 16280093 16361757 16369086 16374267, hotness: undefined, score: 0, children: 2
phoenixz 42 points 2 years ago

Move the ssh port to higher ranges, 30-60000. That alone will stop 99% of the attacks

Disable root logins, now usernames must be guessed too which will make success even lower

Then require SSH keys

At that point it's like being in a nuclear fallout nshelter behind a 3 meter thick steel door and you can hear some zombies scratching on the outside... I'm not worried about any of that shit

path: 0 16282405, hotness: undefined, score: 42, children: 14
joshcodes 13 points 2 years ago

For added funs run an SSH tarpit to fuck with the attackers, something like endlessh.

path: 0 16282405 16283564, hotness: undefined, score: 13, children: 2
phoenixz 9 points 2 years ago

Well yeah, sure, but that doesn't really add to your security and it only costs you work and resources

path: 0 16282405 16283564 16284077, hotness: undefined, score: 9, children: 1
joshcodes 5 points 2 years ago

100% agree, that is a "totally for fun" exercise

path: 0 16282405 16283564 16284077 16284301, hotness: undefined, score: 5, children: 0
null_dot 5 points 2 years ago

This is what I do. Changing the port to a higher number will prevent almost all bots.

I understand that obscurity is not security but not getting probed is nice.

Also ssh keys are a must.

I do log in as root though.

However, I block all IPs other than mine from connecting to this port in my host's firewall. I only need to log in from home, or my office, and in a crisis I can just log in to OVH and add whitelist my IP.

path: 0 16282405 16287962, hotness: undefined, score: 5, children: 1
sugar_in_your_tea 3 points 2 years ago

I do log in as root though.

Don't do that. You're one local piece of malware away from getting your server pwned. Logging in as an unprivileged user at least requires another exploit on the server to get root permissions.

path: 0 16282405 16287962 16299010, hotness: undefined, score: 3, children: 0
StructureOfChaos 5 points 2 years ago

Regarding SSH Keys, I was wondering how you keep your key safe and potentially usable from another client?

path: 0 16282405 16291987, hotness: undefined, score: 5, children: 5
callcc 6 points 2 years ago

Be sure to use a passphrase

path: 0 16282405 16291987 16295559, hotness: undefined, score: 6, children: 1
StructureOfChaos 1 point 2 years ago

Or very strong password

path: 0 16282405 16291987 16295559 16354395, hotness: undefined, score: 1, children: 0
a_postmodern_hat 3 points 2 years ago

You could get a hardware key (like a Yubikey) and authenticate with PIV or GPG.

path: 0 16282405 16291987 16296685, hotness: undefined, score: 3, children: 0
gerowen 2 points 2 years ago

Generate a unique key for each client or device. SSH keys identify devices, not people, so I do not recommend sharing the same key between two different devices.

path: 0 16282405 16291987 16309212, hotness: undefined, score: 2, children: 1
StructureOfChaos 1 point 2 years ago

Well, you might have only 1 main client, but if that hardware fails and need to connect from a temporary client or after a fresh install you're out of your own server...

path: 0 16282405 16291987 16309212 16316148, hotness: undefined, score: 1, children: 0
possiblylinux127 -3 points 2 years ago

Changing the port is a total waste of time

Changing the port is just like putting a picture of a window on your door. Harden SSH properly and don't waste time with security via obscurity

path: 0 16282405 16298519, hotness: undefined, score: -3, children: 2
sugar_in_your_tea 10 points 2 years ago

That's not true.

Security through obscurity isn't real security, sure, but it does a lot to reduce the noise in the logs so you can see the more real attacks. Hardening SSH properly is certainly more important, but changing the port also has value.

path: 0 16282405 16298519 16298925, hotness: undefined, score: 10, children: 0
AustralianSimon 5 points 2 years ago

I think the point behind it is to waste the sniffers time sniffing for ports that it could be using to be making attempts.

Its not a security thing, it's just increasing the cost to snoop.

path: 0 16282405 16298519 16299637, hotness: undefined, score: 5, children: 0
BrianTheeBiscuiteer 34 points 2 years ago

In addition to other advice you could also use SSH over Wireguard. Wireguard basically makes the open port invisible. If you don't provide the proper key upfront you get no response. To an attacker the port might as well be closed.

Here's at least one article on the subject: https://rair.dev/wireguard-ssh/

path: 0 16277198, hotness: undefined, score: 34, children: 2
nekusoul 4 points 2 years ago

Exactly. No root login and no password login will do just fine as basic measures, but after that Wireguard is perfect tool for this, no weird rituals required and also quite useful for any other services you don't want and/or need to expose to the internet as well.

path: 0 16277198 16283283, hotness: undefined, score: 4, children: 1
sugar_in_your_tea 3 points 2 years ago

Just remember that you'll only be able to SSH in w/ a device that's already configured for WireGuard. So if you're at a friend's house and haven't set up your phone to do it yet, you'll be forced to use the VPS console to get in. Make sure this is what you want before you do it.

path: 0 16277198 16283283 16299466, hotness: undefined, score: 3, children: 0
gerowen 31 points 2 years ago

I generally do a few things to protect SSH:

  1. Disable password login and use keys only
  2. Install and configure Fail2Ban
  3. Disable root login via ssh altogether. Just change "permit root login" from "no password" to just "no". You can still become root via sudo or su after you're connected, but that would trigger an additional password request. I always connect as a normal user and then use sudo if/when I need it. I don't include NOPASSWD in my sudoers to make certain sudo prompts for a password. Doesn't do any good to force normal user login if sudo doesn't require a password.
  4. If connecting via the same network or IPs, restrict the SSH open port to only the IPs you trust.
  5. I don't have SSH internet visible. I have my own Wireguard server running on a separate raspberry pi and use that to access SSH when I'm away, but SSH itself is not open to the internet or forwarded in the router.
path: 0 16293857, hotness: undefined, score: 31, children: 1
k_rol 4 points 2 years ago

I vote for wireguard here, I don't expose anything other than game servers to the internet

path: 0 16293857 16299382, hotness: undefined, score: 4, children: 0
Dima 28 points 2 years ago

For security disable password authentication - use public key instead, disable root login via ssh - use sudo or su from another user.

To reduce the number of attempts of others trying to get in change the ssh port and/or set-up fail2ban.

You could also set a firewall rule to only allow ssh from your IP address, if you have a static address at home and only need access from there, or have a way to VPN into your home network. Make sure you have a static address if you do this though, you don't want your IP to change and be left locked out of your server.

path: 0 16275246, hotness: undefined, score: 28, children: 3
sugar_in_your_tea 3 points 2 years ago

You could also set a firewall rule to only allow ssh from your IP address

You can also broaden this to a region. You may still want to access SSH from various places around your country (e.g. when visiting family or friends), but likely won't ever need to from most of the rest of the world, so block everything except IPs from your region (or regions you care about, e.g. any VPSs you have).

path: 0 16275246 16276010, hotness: undefined, score: 3, children: 2
7toed 1 point 2 years ago

No if you're doing that, use a VPN through your firewall. Local traffic is a fair exception as this can only ever be a device on your network, but that depends on your threat model (as those local devices could be compromised). Opening to "your region's" IP range opens you to a lot more than LAN access..

path: 0 16275246 16276010 16276293, hotness: undefined, score: 1, children: 1
sugar_in_your_tea 4 points 2 years ago

Sure. I'm just assuming that you'd want to access it from areas in your region, like at a friend or family member's house, work, coffee shop, etc. This is especially true if you or one of them has DHCP from their ISP. You only really need to whitelist a handful of IP ranges to get most of the benefit of IP-based blocking and most of the convenience of not having a block.

If you only ever truly need it at home, then sure, do that. In fact, for something like SSH, you could probably create a Wireguard endpoint that's accessible almost anywhere and connect to that before logging in via SSH.

My point is to not make it more restrictive than you need, otherwise it'll just be frustrating and you'll end up disabling whatever protections you have. You can get a lot of value with a broad ban on troublesome areas (e.g. I don't visit most of the places OP has in their logs, so those would be banned), and then fine-tune the rest w/ something like fail2ban.

I use my ISP's firewall rule to whitelist regions I'm interested in, such as:

  • data centers where I have services
  • my town and my friends' and families' towns
  • the town near where I work

I have different rules by port, so SSH is a lot more restricted than HTTP(s).

path: 0 16275246 16276010 16276293 16276643, hotness: undefined, score: 4, children: 0
RonnyZittledong 24 points 2 years ago

Disable passwords and use public private keys and don't worry about it

path: 0 16274876, hotness: undefined, score: 24, children: 2
JubilantJaguar 9 points 2 years ago

This is the only answer you need to read. It's a non-problem if you just do this, and there's no reason not to do it.

path: 0 16274876 16279466, hotness: undefined, score: 9, children: 1
callcc 1 point 2 years ago

Agreed!

path: 0 16274876 16279466 16295603, hotness: undefined, score: 1, children: 0
zr0 21 points 2 years ago path: 0 16283194, hotness: undefined, score: 21, children: 6
om1k 13 points 2 years ago

did you mean crowdsec instead of crowdstrike?

path: 0 16283194 16285718, hotness: undefined, score: 13, children: 2
zr0 4 points 2 years ago

Fml… yes, I meant CrowdSec. Thanks for the hint

path: 0 16283194 16285718 16288578, hotness: undefined, score: 4, children: 0
whodatdair 1 point 2 years ago

Vietnam stare

path: 0 16283194 16285718 16288884, hotness: undefined, score: 1, children: 0
sugar_in_your_tea 4 points 2 years ago

harden sshd

More details:

  • require keys to login
  • don't allow login as root

That should be plenty, but you could go a bit further and restrict the types of algorithms allowed (e.g. disallow RSA if you're worried about quantum attacks). For this, I recommend a subtractive config (e.g. HostbasedAcceptedAlgorithms=-rsa-*). This is way over the top since an attacker is unlikely to attack the cipher directly, but it could be part of an attack.

path: 0 16283194 16299308, hotness: undefined, score: 4, children: 0
dev_null 3 points 2 years ago

Your answer to "how to harden SSH?" is "harden SSH"?

I know your two other points gave concrete suggestions, but it's pretty funny you suggested to "harden sshd" when that is what OP is asking how to do.

path: 0 16283194 16294923, hotness: undefined, score: 3, children: 1
zr0 1 point 2 years ago

Yeah, I see your point. No use to repeat the same you can read in other comments or in those 274772 guides online. I was trying to imply to just generally harden ssh because then brute-force attempts should be no issue, unless you log everything and the disk space gets maxed out :D

path: 0 16283194 16294923 16299967, hotness: undefined, score: 1, children: 0
irmadlad 21 points 2 years ago

OP, here is what I do. It might seem overboard, and my way doesn't make it the best, or the most right, but it seems to work for me:

  • Fail2ban
  • UFW
  • Reverse Proxy
  • IPtraf (monitor)
  • Lynis (Audit)
  • OpenVas (Audit)
  • Nessus (Audit)
  • Non standard SSH port
  • CrowdSec + Appsec
  • No root logins
  • SSH keys
  • Tailscale
  • RKHunter

The auditing packages, like Lynis, will scour your server, and make suggestions as to how to further harden your server. Crowdsec is very handy in that it covers a lot of 'stuff'. It's not the only WAF around. There is Wazuh, Bunkerweb, etc. Lots of other great comments here with great suggestions. I tend to go overboard on security because I do not like mopping up the mess after a breach.

ETA: just looked up one of your attackers:

218.92.0.201 was found in our database! This IP was reported 64,044 times. Confidence of Abuse is 100%: ISP CHINANET jiangsu province network Usage Type Fixed Line ISP ASN AS4134 Domain Name chinatelecom.cn Country China City Shanghai, Shanghai

busy little cunts.

path: 0 16280971, hotness: undefined, score: 21, children: 4
db0 6 points 2 years ago

No Port-knocking? Amateurs! /s

path: 0 16280971 16285196, hotness: undefined, score: 6, children: 0
sugar_in_your_tea 2 points 2 years ago

It's absolutely overboard, and you can get 99% of the way there with this:

  1. WireGuard config (Tailscale in your case)
  2. Bind SSH to WireGuard IP only (so no public SSH port)
  3. SSH keys only, and disable root login over SSH

That will require breaking WireGuard and openSSH's key-based authentication, which just isn't happening. The rest looks like mostly auditing. Even a firewall isn't necessary if no ports are accessible anyway (i.e. everything only accessible over Tailscale), and you can just configure iptables to block everything on the WAN IP and call it a day.

path: 0 16280971 16299425, hotness: undefined, score: 2, children: 2
irmadlad 2 points 2 years ago

sugar_in_your_tea @sh.itjust.works

It's nice to be commented by someone famous.

Open up the window, let some air into this room I think I'm almost chokin' from the smell of stale perfume And that cigarette you're smokin' 'bout scare me half to death Open up the window, sucker, let me catch my breath

path: 0 16280971 16299425 16300652, hotness: undefined, score: 2, children: 1
sugar_in_your_tea 1 point 2 years ago

Mama told me not to come.

Fun fact, my usernames on Reddit (I would cycle them every couple of years) were all Three Dog Night lyrics, so I continued the theme on Lemmy.

Thanks for noticing. 😀

path: 0 16280971 16299425 16300652 16300915, hotness: undefined, score: 1, children: 0
bizdelnick 20 points 2 years ago

The best way is to disable password login and use SSH keys only. Any further steps are not required, but you may additionally install fail2ban or sshguard.

path: 0 16292375, hotness: undefined, score: 20, children: 0
kylian0087 19 points 2 years ago

Configure the firewall with a IP whitelist to only allow connections to ssh be made from your home IP.

Other then that, disable password logon for ssh and setup up key based authentication.

path: 0 16288367, hotness: undefined, score: 19, children: 1
sugar_in_your_tea 7 points 2 years ago

Agreed, but be careful about the whitelist. If your home IP changes, you'll be locked out until you update it, so you should consider an IP range if that's a possibility for you. Likewise, if you'll be accessing it from multiple locations (say, a family member's house), then make sure to add those as well.

path: 0 16288367 16298886, hotness: undefined, score: 7, children: 0
possiblylinux127 18 points 2 years ago

Use key based auth only and then run ssh-audit.

path: 0 16298457, hotness: undefined, score: 18, children: 0
Arghblarg 17 points 2 years ago

There's a dedicated tool named sshguard which works nicely.

path: 0 16285160, hotness: undefined, score: 17, children: 0
tomsh 15 points 2 years ago

In addition to what others say, I also have ntfy notifications on successful login.

path: 0 16296878, hotness: undefined, score: 15, children: 2
joseandres42 6 points 2 years ago

That's genius. I'll do the same from now on.

path: 0 16296878 16298103, hotness: undefined, score: 6, children: 1
tomsh 10 points 2 years ago path: 0 16296878 16298103 16298909, hotness: undefined, score: 10, children: 0
Realitaetsverlust 13 points 2 years ago

You don't. This is normal. Ensure key-only auth, ensure you do not login directly as root, maybe install fail2ban and you're good. Some people move the port to a nonstandard one, but that only helps with automated scanners not determined attackers.

You could look into port-knocking if you want it really safe.

path: 0 16276122, hotness: undefined, score: 13, children: 1
suicidaleggroll 4 points 2 years ago

Some people move the port to a nonstandard one, but that only helps with automated scanners not determined attackers.

While true, cleaning up your logs such that you can actually see a determined attacker rather than it just getting buried in the noise is still worthwhile.

path: 0 16276122 16283227, hotness: undefined, score: 4, children: 0
Waryle 13 points 2 years ago

You can look up for:

  • Setting up max authentication attemps per connection -> slows up a lot brute force attacks. If your password is strong enough, that's already a big step to secure your server.
  • Generate SSH Keys and disable password authentication -> do this only if you're connecting through the same devices, because you won't be able to connect from any device that has not being set up. Personally I don't use this because I want to be able to access my server even if I'm not home and without my laptop
  • Set up Crowdsec -> it's a local service which scans logs and will block access to any suspicious IPs. It also relies on a crowdsourced list of IPs that are identified as threat and will preventively block them
path: 0 16275003, hotness: undefined, score: 13, children: 3
LlilL 1 point 2 years ago

Is this an alternative/replacement to fail2ban or something you would use along with f2b?

path: 0 16275003 16275544, hotness: undefined, score: 1, children: 2
CausticFlames 5 points 2 years ago

You could technically still use it alongside f2b, but in my experience Crowd-Sec seems to do a better job and can do the same things.

path: 0 16275003 16275544 16276087, hotness: undefined, score: 5, children: 1
LlilL 2 points 2 years ago

Thank for that! You just turned a student onto a new tool to play with.

path: 0 16275003 16275544 16276087 16285580, hotness: undefined, score: 2, children: 0
EarMaster 8 points 2 years ago

For a general guide on how to make ssh more secure I stick to https://www.sshaudit.com/

You can check your config and they also provide step by step guides for several distros...

path: 0 16276692, hotness: undefined, score: 8, children: 0
troed 8 points 2 years ago

A few replies here give the correct advice. Others are just way off.

To those of you who wrote anything else than "disable passwords, use key based login only and you're good" - please spend more time learning the subject before offering up advice to others.

(fail2ban is nice to run in addition, I do so myself, but it's more for to stop wasting resources than having to do with security since no one is bruteforcing keys)

path: 0 16276511, hotness: undefined, score: 8, children: 5
nekusoul 6 points 2 years ago

Eh, while I agree that some recommendations are dodgy at best, I'll argue that Wireguard is not only adding to security, it also makes Fail2Ban obsolete. Due to the way it works, you'll completely hide the fact that you're even running a SSH server at all, and this includes even Wireguard itself. More importantly though, it's pretty much impossible to set up Wireguard in an insecure way, whereas SSH provides you with plenty of footguns. You're not risking locking yourself out either.

Also, security comes in layers.

path: 0 16276511 16283726, hotness: undefined, score: 6, children: 1
sugar_in_your_tea 3 points 2 years ago

You’re not risking locking yourself out either.

In a VPS, you should always be able to fall back to the web console. So locking yourself out shouldn't be a major concern.

path: 0 16276511 16283726 16283752, hotness: undefined, score: 3, children: 0
sugar_in_your_tea 2 points 2 years ago

There's more to it than that.

I recommend geoip blocking anything outside of your expected operating regions in addition to using key-based logins. iptables operates at a lower level in the network stack than SSH, so the vulnerability surface is a lot lower, and blocking before something actually looks at the packets cleans up the logs. This is huge because it makes it a lot more obvious when there's a legitimate attack.

Cover yourself with layers:

  1. block obviously bad packets at the firewall level
  2. eliminate insecure modes of login (only allow key-based login)
  3. something like fail2ban to ban the few who make it through 1 & 2
  4. use a secure root password so if someone does get in, they're less likely to get root access
  5. have your services run as non-privileged users to limit issues if something gets compromised

If you only do one thing, it should be only allowing key-based logins. If you do two, run SSH on a non-standard port or set up geoip blocking (second is more work, but a lot more effective).

path: 0 16276511 16283231, hotness: undefined, score: 2, children: 2
troed 2 points 2 years ago

Still no. Here's the reasoning: A well known SSHd is the most secure codebase you'll find out there. With key-based login only, it's not possible to brute force entry. Thus, changing port or running fail2ban doesn't add anything to the security of your system, it just gets rid of bot login log entries and some - very minimal - resource usage.

If there's a public SSHd exploit out, attackers will portscan and and find your SSHd anyway. If there's a 0-day out it's the same.

(your points 4 and 5 are outside the scope of the SSH discussion)

path: 0 16276511 16283231 16290055, hotness: undefined, score: 2, children: 1
sugar_in_your_tea 2 points 2 years ago

It's also one of the biggest targets for attack. Here's a somewhat recent CVE and here is another. Staying on top of security patches is absolutely critical, and many don't do that.

The best security practice is to layer your protections.

your points 4 and 5 are outside the scope of the SSH discussion

They're not about SSH, sure, but they are relevant to securing a system to remote access. Always assume your security infra will be compromised and plan accordingly. Generally speaking, the more layers, the better.

path: 0 16276511 16283231 16290055 16296392, hotness: undefined, score: 2, children: 0
plz1 7 points 2 years ago

Does SSH have to be your only way? Could you deploy something like Tailscale? Can you restrict the allowed IP ranges on SSH with a firewall rule?

path: 0 16292209, hotness: undefined, score: 7, children: 0
hemmes 7 points 2 years ago

What VPS are you using?

You should be able to setup a firewall, blocking all access to the SSH port. Then setup a VPN so that only you can access via SSH after making your VPN connection.

If you connect via a static IP, you can also create an ACL for the VPN connection just in case. You can set an ACL for the SSH port forward rule directly as well, but I don’t like that personally. I prefer keeping things behind the VPN.

path: 0 16274955, hotness: undefined, score: 7, children: 18
Voroxpete 3 points 2 years ago

This is the correct answer. Never expose your SSH port on the public web, always use a VPN. Tailscale, Netmaker or Netbird make it piss easy to connect to your VPS securely, and because they all use NAT traversal you don't have to open any ports in your firewall.

Combine this with configuring UFW on the server (in addition to the firewall from the VPS provider - layered defence is king) and Fail2Ban. SSH keys are also a good idea. And of course disable root SSH just in case.

With a multi-layered defence like this you will be functionally impervious to brute force attacks. And while each layer of protection may have an undiscovered exploit, it will be unlikely that there are exploits to bypass every layer simultaneously (Note for the pendants; I said "unlikely", not "impossible". No defence is perfect).

path: 0 16274955 16275414, hotness: undefined, score: 3, children: 16
troed 5 points 2 years ago

This is not "the correct answer". There's absolutely nothing wrong with "exposing" SSH.

path: 0 16274955 16275414 16276559, hotness: undefined, score: 5, children: 15
Voroxpete -6 points 2 years ago

You seem like a fan of the "pull out" method.

path: 0 16274955 16275414 16276559 16276944, hotness: undefined, score: -6, children: 14
callcc 4 points 2 years ago

Public ssh is completely fine as long as you use key based auth only and keep your sshd up to date. Stop spreading bullshit.

path: 0 16274955 16275414 16276559 16276944 16278364, hotness: undefined, score: 4, children: 11
troed 2 points 2 years ago

Feel free to argue with facts. Hardening systems is my job.

path: 0 16274955 16275414 16276559 16276944 16277261, hotness: undefined, score: 2, children: 1
peregus 3 points 2 years ago

Exactly, this I what alI do!

path: 0 16274955 16275464, hotness: undefined, score: 3, children: 0
lefixxx 7 points 2 years ago

Change the port.

path: 0 16300316, hotness: undefined, score: 7, children: 0
aeternum 6 points 2 years ago
path: 0 16310308, hotness: undefined, score: 6, children: 0
lichtmetzger 6 points 2 years ago

Use knockd and iptables, gone are the fucking bots.

path: 0 16309439, hotness: undefined, score: 6, children: 0
AceSLS 5 points 2 years ago

Like others said, disable password auth and setup auth keys instead.

Bonus points for moving the ssh port, using fail2ban and also setting up a tarpit with something like endlessh.

If you wanna go extreme use Wireguard to connect to your server and only allow ssh over wireguard in your firewall.

path: 0 16275098, hotness: undefined, score: 5, children: 0
cecilkorik 5 points 2 years ago

fail2ban is mandatory equipment for any ssh server accessible to the public especially on its default port. It's highly configurable, but the default settings will do fine at making it statistically impossible for any user or password to be brute forced.

path: 0 16274900, hotness: undefined, score: 5, children: 2
cron 3 points 2 years ago

I don’t really get the love for fail2ban. Sure, it helps keep your logs clean, but with a solid SSH setup (root disabled, SSH keys enforced), I’m not bothered by the login attempts.

path: 0 16274900 16278505, hotness: undefined, score: 3, children: 1
sugar_in_your_tea 3 points 2 years ago

You should be. Most of it's noise, but if there's a serious attack, you'll appreciate clean logs.

I think fail2ban is nice as like a third or fourth layer of defense. In order of my priorities:

  1. key-only login, root login completely disabled
  2. solid root password, and user privilege separation (have each service use its own user)
  3. geoip bans - if you never plan to support clients in a given region, block it at the firewall level (or better yet, whitelist the handful of regions you care about); I do this by port, so SSH gets a much more restricted set of allowed regions than HTTPS
  4. fail2ban - especially if you have a relatively large whitelist
  5. only access SSH over a Wireguard VPN - Wireguard doesn't show up in port scans, and SSH can bind to the VPN host instead of 0.0.0.0, so the ability to login via SSH will be completely hidden

If you're not going to do 3-5, at least change the default SSH port to cut down on log noise.

path: 0 16274900 16278505 16283418, hotness: undefined, score: 3, children: 0
boblemmy 5 points 2 years ago

change port + fail2ban + totp (google-authenticator)

path: 0 16309911, hotness: undefined, score: 5, children: 0
Appoxo 4 points 2 years ago

You could limit the firewall to IP range(s) of your domestic (and other places of interest like work) connection.
This way they won't come even close to even logging in.
And then you could do the other hardening on top.

path: 0 16316424, hotness: undefined, score: 4, children: 0
demesisx 4 points 2 years ago

If you can use another method, disabling SSH entirely would do it. ;)

This is how Talos Linux achieves best-in-class security properties.

https://www.siderolabs.com/...

path: 0 16276753, hotness: undefined, score: 4, children: 0
Treczoks 3 points 2 years ago

Move away from port 22, and 90% vanishes. Move it up to a port in the five digit range, and you will rarely see them.

path: 0 16328198, hotness: undefined, score: 3, children: 0
tal 3 points 2 years ago

If you use keys or strong passwords, it really shouldn't be practical for someone to brute-force.

You can make it more-obnoxious via all sorts of security-through-obscurity routes like portknocking or fail2ban or whatever, or disable direct root login via PermitRootLogin, but those aren't very effective compared to just using strong credentials.

path: 0 16274913, hotness: undefined, score: 3, children: 0
sugar_in_your_tea 3 points 2 years ago

One of the simplest is geoip blocks. Here's an article using iptables, and there may be a nicer way w/ whatever firewall you're using.

For reference, here are the areas I see in your logs (using this service):

  • 218.92.0.201 - China
  • 162.142.125.122 - US (Michigan)
  • 45.79.181.223 - US (New Jersey)
  • 118.25.174.89 - China
  • 92.118.39.73 - Romania
  • 98.22.89.155 - US (Nebraska)
  • 75.12.134.50 - US (Tennessee)
  • 165.140.237.71 - US (Washington)
  • 65.49.1.29 - US (California)

If you don't expect valid users to come from those areas, block them. A lot of those in the US are probably from VPN users, so be careful if people are using a VPN to connect to your services.

If you can do it w/ iptables, it'll be a lot more efficient than doing it at the application layer. I also recommend using something like fail2ban to block individual IPs within regions you care about to get any stragglers that make it through the first tier of blocks. Since this is a VPS, you can also check what firewall settings your provider has and see if you can configure it there so it doesn't make it to your instance in the first place.

path: 0 16276267, hotness: undefined, score: 3, children: 3
someacnt 1 point 2 years ago

Thanks a lot! Geoblocking makes a lot of sense, will try!

path: 0 16276267 16284159, hotness: undefined, score: 1, children: 2
sugar_in_your_tea 2 points 2 years ago

I highly recommend using key-based SSH authentication exclusively for all users on your server, and disallow root login as well.

Geoblocking mostly cuts down on the spam, but also constrains where an actual attack can come from. If there's some kind of zero-day attack on SSH, this will dramatically reduce the risk you're hit.

path: 0 16276267 16284159 16284202, hotness: undefined, score: 2, children: 1
someacnt 2 points 2 years ago

Fortunately my VPS (oracle) has set SSH authentication to be default. Disallowing root login sounds good, gotta try that as well.

path: 0 16276267 16284159 16284202 16284274, hotness: undefined, score: 2, children: 0
EncryptKeeper 3 points 2 years ago

https://www.crowdsec.net/

Take the concept of Fail2Ban and add in a community blocklist of thousands of IPs so that you’re blocking not only IPs that have attacked you, but others as well.

It’s neat because they have a number of collections you can download from the community that include readymade parsers for other kinds of logs, and other attack scenarios you can guard against. For example, if you run Nginx or Caddy as webservers on that machine, you can download associated collections for each that can parse your web access log files and ban IPs based on IPs probing your web server for unprotected admin panels, or abusive AI crawlers.

You can even write your own scenarios. I wrote one that immediately blocks you after just one attempt to log in using an account like root, admin,adm,administrator, etc.

path: 0 16277315, hotness: undefined, score: 3, children: 1
irmadlad 3 points 2 years ago

+1 for Crowdsec

path: 0 16277315 16281525, hotness: undefined, score: 3, children: 0
slazer2au 2 points 2 years ago

I assume you have root login denied in your ssh config, other things would be having fail2ban and some geofencing (blocking IPs from countries you know you are never going to log in from).

path: 0 16274921, hotness: undefined, score: 2, children: 0
bla_bla_bla 1 point 2 years ago
path: 0 16276097, hotness: undefined, score: 1, children: 0
Enkers 1 point 2 years ago

I think VPN is the proper way to go about this, but another method is to do port knocking with fkwnop so your SSH port won't respond until the host receives a magic packet.

path: 0 16276177, hotness: undefined, score: 1, children: 0
ShortN0te 1 point 2 years ago

Do you want to prevent brute forcing or do you want to prevent the attack getting in?

If you want to prevent brute forcing then software like fail2ban helps a little, but this is only a IP based block, so with IPv6 this is not really helpfull against a real attack, since rotating IP addresses is trivial. But still can slow down the attacker. Also limiting the amount of sessions and auth tries does significantly slow down the attacker.

If you just want to not worry about it set strong passwords, and when it is a multi user system where other ppl might access it, configure Public Key Auth so you can be sure the other users have strong passwords (or keys in this case) to authenticate.

With strong passwords or keys it is basically impossible to brute force your way in with ssh.

path: 0 16275623, hotness: undefined, score: 1, children: 0
unlogic 1 point 2 years ago
path: 0 16277374, hotness: undefined, score: 1, children: 0
csm10495 1 point 2 years ago

Bad eh security advice: use an alternative ssh port. Lots of actors try port 22 and other common alternatives. Much fewer will do a full port scan looking for an ssh server then try brute forcing.

path: 0 16276392, hotness: undefined, score: 1, children: 0
neidu3 1 point 2 years ago

I use fail2ban and a non-standard SSH port. 99% of this junk disappears if you run sshd on port 9.

Also, disable password for login - Only use keys.

path: 0 16276744, hotness: undefined, score: 1, children: 0
surph_ninja 1 point 2 years ago

Does it need to be exposed to the internet? Putting it behind a vpn would be best.

Besides that, just make sure only the users you need to have access to ssh logins, and use keys for extra hardening. Keep your system updated. Limit that system’s access to other systems on your network, so if it is compromised, they can’t use it as a pivot point for the rest of your setup.

The other commenter’s suggestion of fail2ban is also solid.

path: 0 16276078, hotness: undefined, score: 1, children: 0
sntx 1 point 2 years ago path: 0 16292386, hotness: undefined, score: 1, children: 0
Fedditor385 0 points 2 years ago

Setup fail2ban

path: 0 16276980, hotness: undefined, score: 0, children: 0
callcc -4 points 2 years ago

Welcome to the internet! Your system will get probed. Make sure you run as little as possible services on open ports and only high quality ones such as OpenSSH. Don't freak out because of your logs. You're fine as long as your system is up to date and password login disabled! Don't listen to the fail2ban or VPN crowd. Those are only snake oil.

A VPN is probably just as (in)secure as OpenSSH. There is no gain in complicating things. OpenSSH is probably one of the most well tested code for security around.

path: 0 16278533, hotness: undefined, score: -4, children: 0
selfhosted
selfhosted

@lemmy.world

login for more options
62947
6792
5617

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

  1. Be civil.

  2. No spam.

  3. Posts are to be related to self-hosting.

  4. Don't duplicate the full text of your blog or readme if you're providing a link.

  5. Submission headline should match the article title.

  6. No trolling.

  7. Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.

  8. AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

go to feed...