Docker or Proxmox? Something else entirely?

a year ago by JeanValjean to c/selfhosted

New server has been acquired. Debian 13 has been installed.

GS308EP switches have been acquired and installed.

Now, I'm working to migrate to the new machine. 3 1/2 years ago when I started futzing with Docker, I sorta followed guides and guessed, abused it trying to make it do things it wasn't designed for, and flipped switches I likely shouldn't have flipped, so the set up is more than a little shabby.

As a result, I'll likely end more redeploying than migrating the containers.

So rather than go forward with Docker blindly, I want to reassess whether I shouldn't look into Proxmox, LXC, or Podman instead of Docker, or maybe something else entirely?

Work is just about done dumping ESX for Nutanix, but both of those seem overkill for my needs.

Of course the forums for any of the solutions make their own out to be the best thing since sliced bread and the others useless, so I'm hoping to get a more nuanced answer here.

load all comments

glizzyguzzler 23 points a year ago

If you’ve got Debian already installed, I cannot resist advocating for Incus (stable branch from Zabbly repo with web ui https://blog.simos.info/...) in lieu of proxmox. Does the same thing but you don’t have to rip out the kernel Debian uses.

With Debian 13 you have access to podman quadlets, use that for any non-vm needs. The ease of docker compose files easily removes reason for programs in LXC containers, and podman removes reason for docker in an LXC. LXC is left only for programs that aren’t containerized. VMs for security DMZ. Podman for bulk of stuff you want.

Good luck!

save

path: 0 18921795, hotness: undefined, score: 23, children: 7

tofubl 4 points a year ago

I was in a similar spot not too long ago, setting up a firewall and general network box. I was going to go with Proxmox but a fellow Lemmy guy strongly advocated for Incus on top of vanilla Debian. I was intrigued and ended up going for it. Learned a lot about networking with systemd (bridging, IP assignment and so on) for things I could have gotten for free in Proxmox (literally a few clicks), and had to fight Incus to work with a FreeBSD VM for Opnsense, but I love the setup now. Pure debian with a few Incus VMs and Docker inside of those as needed. So clean!

save

path: 0 18921795 18924704, hotness: undefined, score: 4, children: 6

koala 6 points a year ago

Came in here to mention Incus if no one had.

I love it. I have three "home production" servers running Proxmox, but mostly because Proxmox is one of very few LTS/comercially-supported ways to run Linux in a supported way with root (and everything else on ZFS). And while its web UI is still a bit clunky in places, it comes in handy some times.

However, Incus automation is just... superior. incus launch --vm images:debian/13 foo, wait a few seconds then incus exec foo -- bash and I'm root on a console of a ready-to-go Debian VM. Without --vm, it's a lightweight LXC container. And Ansible supports running commands through incus exec, so you can provision stuff WITHOUT BOTHERING TO SET UP ANYTHING.

AND, it works remotely without fuss, so I can set up an Incus remote on a beefy server and spawn VMs nearly transparently. + incus file pull|push to transfer files.

I'm kinda pondering scripting removal of the Proxmox bits from a Proxmox install, so that I just keep their ZFS support and run Incus on top.

save

path: 0 18921795 18924704 18926028, hotness: undefined, score: 6, children: 2

glizzyguzzler 3 points a year ago

I’m looking at Opnsense on an Incus VM soon, what was your fight there? Good to know what I’ll hit ;)

Agreed on that path - some networking (like mimicking proxmox’s bridge connections which give VMs their own MAC/IP) takes effort to find the solution. But the basic LXC/VM-shares-your-IP works super easily and the script ability is great. Plus it doesn’t feel like a yoke on your system that is heavy and drives it, but just another application! I feel it’s close enough, and when you get it where you want it, it’s perf. I assume they’ll get “one click” solutions for the harder stuff baked in as they get more attention and traction.

save

path: 0 18921795 18924704 18929433, hotness: undefined, score: 3, children: 2