I hadn't heard of that before now! Very interesting. Through very briefly looking in to podman its basically a docker alternative - so if i rephrased my questions to "host+container vs proxmox" your answer is the former. How is the back up solution? As a few people have mentioned that as a pro of proxmox?
I run everything on Btrfs (but zfs is also good), which allows filesystem level snapshotting. Together with a tool like btrbk (packaged in most distros) that makes for a very nice automated backup system. Zfs has the function of btrbk built in directly.
Just saw a post about podman here you may find interesting/ want to input on
I run OpenSUSE tumbleweed as my daily driver with btrfs and the snapshots have saved me from breaking updates. I think i'll need to experiment at this stage... (but i guess thats exactly what a homelab is for!)
I think I have to little experience to answer everything. But at some point I decided I will always run Proxmox (even if you then have only one VM or CT there).
Why? Because it gives me good backup (do a snapshot), I can move that VM/CT in a second somewhere else, gives me a layer of monitoring,...
I use option 1 and would do it again. Can recommend. Proxmox as the base OS is just so flexible. Then a VM with Debian which runs docker and most of my docker containers.
Some selected docker containers run in there own VM for segregation. (For example troubleshooting zigbee2mqtt required rebooting the VM, so it had its own VM. Wireguard runs in its own VM because (at the time) a newer kernel was needed. nginx runs as an LXC cause I wanted some integration that wasn't straight forward with Docker.
For the most part Docker is probably easier than LXC, but for Jellyfin I'll go with LXC for GPU accelerated transcoding. (Which can be a pain in VMs)
Proxmox doesn't have much overhead compared to bare metal, (unless you run in to a bug or issue, and don't need GPU Acceleration) so for me the flexibility of Proxmox is 100% worth it.
Also check out Proxmox Backup Server, which makes VM and LXC backups stupid easy
With proxmox you can also start out with a VM with all you're current docker images like a drop-in replacement, then gradually move them to LXC at your own pace. (Though I don't see any reason to do so except the learning exercise)
If you do go with Proxmox, the recommendation is to install docker in a VM. Do not install Docker on the proxmox host directly
Second the proxmox suggestion, but I’m still on the fence myself between VM+docker versus individual LXCs.
Nothing more convenient than a docker-compose pull && docker-compose up -d when everything is defined in a compose file.
The simplicity and ubiquity of docker compose is exactly why I haven't decided to use LXCs but instead have proxmox and one VM that handles the majority of my docker containers which I view / manage with Portainer (though not fully utilizing stacks). Proxmox is great for VM management, resource allocation, and backup and docker is great to keep overhead low, quickly recreate services, and is more commonplace these days than something akin to Vagrant
I do really like this pro too and at this point am very used to it and it works for me. I'll see what the overhead is with Proxmox and go from there. At the end of the day, hopefully I dont need to squeeze every last drop of RAM out of my system anymore so if it works and I can run more services then its a win
I switched from Docker to Proxmox and I never want to go back. Each server has its own CT and I can even run VMs on the side. Backups have already been mentioned.
I took a look at this real quick too How to Set Up Docker Containers in Proxmox. With Proxmox you're already creating a container and vm object but from this guide, you're installing Ubuntu then installing docker on top of that for containers. So I dont think nesting containers on containers is a good idea either.
This forum post on proxmox's site seems to show how you can run docker in 3 options: run Docker in VM, run Docker in LXC, or run Docker directly on the PVE node.
The other point of concern is moving the docker images to a new host. Also if moving away from docker to their own LXC migrating the data and re-establishing your network of apps.
Hello, I use Proxmox on a weaker MiniPC and have no problems with it at all. You can install Docker in LXC containers via container nesting and then have even less overhead than running Docker in a VM in Proxmox. Personally, I find the possibility of setting up VMs in parallel and having disposable systems with just a few clicks very appealing. Proxmox also offers some support for backups. But I haven't tried that myself yet. Maybe that's something you would also be interested in for your Docker containers.
Depending on what you need. If you plan to run everything via docker, install docker+portainer on the host and manage your docker instances via portainer. Proxmox is great with lot of functionality. If you don't plan to use proxmox as a hypervisor, avoid the overhead and just go with portainer and docker on bare metal.
Thanks. I guess that is ultimately the question I need to ask myself - will I need a VM in future. Looking at the other comments it seems the overhead isnt that high (though no-one gave me a number). In the event I did need a VM its ready to go rather than being stuck with a messy solution to fire one up
That's true. I use quite a few VMs but I have enterprise grade servers in my homelab. If I were to use a consumer grade PC, I would probably just focus on docker containers. This way I can get maximum usage out of it. I am totally pro proxmox as it has been the best hypervisor I have used for my home lab and with Proxmox backup server running, the backups are seamless.
I am currently struggling through to change over from a SBC running Ubuntu with everything in Docker to a mini PC running Proxmox and utilizing Docker. I think it will depend on your pain tolerance and existing linux-fu.
Where I am totally comfortable doing as another poster said and spinning up Fedora server and plugging along, I wanted to give proxmox a go in order to have access to all that extra potential with VM's (home assistant, pfsense/opnsense, etc)... But multiple times I have been very close to saying fuck it and binning the whole Proxmox set up.
I am not a power user, so everything is a steep learning curve, but the complexities of unprivileged containers running docker and permissions on the host nearly broke me.
But the nice thing with Proxmox is being able to fuck around and try different things, and when it doesn't work just burn the VM or container and not have to reinstall. I ended up setting up docker in a privileged container (which the internet would make it sound like doing so is committing a deadly soon) just so I could get my services up and running through docker without issue, and then I can dick around with "properly" setting up an unprivileged container that can accomplish the same.
But the simplicity of going back to Fedora + Docker + BTRFS snapshots will always tempt me, I think.
thanks for using Leebra!
go to feed...
Fedora server with Podman and Quadlet Systemd scripts is the best option for small scale self-hosters IMHO. I would not go down the VM route, waste of resources and overly complicated.
save