Kopia, as an alternative to Borg within single threaded, resource constrained systems?

a month ago by kiol to c/linuxadmin

Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included. - kopia/kopia

Been looking into resource constraining Kopia through systemd to prevent the Go binary from eating through too many resources. Anyone running a system like this?

victorz 3 points a month ago

How come you don't want it to use resources? Unused RAM is wasted RAM etc. I'm sure you have legitimate reasons, I'm just curious what they might be. 😁

save

path: 0 25701749, hotness: undefined, score: 3, children: 6

kiol 2 points a month ago

I was considering trying it on resource constrained machines, with 2 - 8gb ram available. Been looking at Kopia or Borg. Borg is limited to single core.

save

path: 0 25701749 25705197, hotness: undefined, score: 2, children: 0

kiol 1 point a month ago

I'm concerned about resource usage spikes while trying to run other services on same device.

save

path: 0 25701749 25705212, hotness: undefined, score: 1, children: 4

victorz 2 points a month ago

Ah okay. Like in the sense that those other services might not survive, or just performance degradation?

Cause I'm thinking otherwise I'd just run the backup overnight or something. 😅 But that might still not be feasible of course.

save

path: 0 25701749 25705212 25705986, hotness: undefined, score: 2, children: 3

kiol 1 point a month ago

Performance degradation, but figured I'd ask outside opinions. So far, sounds like performance is solid relative to resource usage. I'll give it a go and play around with letting it run mult-core vs single core.

save

path: 0 25701749 25705212 25705986 25707941, hotness: undefined, score: 1, children: 2

victorz 2 points a month ago

Cool cool, I hope you find a configuration that works satisfactorily for your needs!

save

path: 0 25701749 25705212 25705986 25707941 25708085, hotness: undefined, score: 2, children: 1

deffard 3 points a month ago

I have run it for several years overnight without issue, backing up around 10GiB, but can only vouch for the stability of the solution and not its behaviour under constrained resources. I had come from duplicati which always ended up with a corrupt database I could not restore from. This project has been a solid replacement, so I would recommend giving it a go.

save

path: 0 25692850, hotness: undefined, score: 3, children: 2

kiol 3 points a month ago

Have you tried running from CLI, or only through the server?

save

path: 0 25692850 25694236, hotness: undefined, score: 3, children: 1

deffard 2 points a month ago

I run the schedule through the server, which also allows you to browse the snapshots, restore individual files from them, and validate what has been backed up.

save

path: 0 25692850 25694236 25705967, hotness: undefined, score: 2, children: 0

scrion 3 points a month ago

In general, if you want to decrease CPU and memory usage, the first and foremost thing you should do is decrease parallelism via the kopia settings (i. e. either as a global policy for scheduled backups, or directly as parameter when creating a snapshot). No idea about kopia-ui, but the settings are probably exposed there? Not a ui user.

You can also decrease resource usage by tweaking compression settings, or disabling compression completely.

Once that is done, you can experiment with hard limits. I'm not sure how resilient kopia is in the presence of those, or how performance might suffer - all I can say is I backed up 2TB of data just 3 or 4 days ago using reasonable settings, and it did not tax the host overly much.

save

path: 0 25694274, hotness: undefined, score: 3, children: 1

kiol 1 point a month ago

What sort of system specs are you working with?

save

path: 0 25694274 25705204, hotness: undefined, score: 1, children: 0