backup to multiple destinations at the same time

[scoddy]

In order to prevent backup corruption due to a faulty destination, restic could get the feature to backup to two destinations in one run.

[fw42]

why in one run? why can't you just backup twice with different destinations?

[fd0]

If the underlying data changes, you might not get exactly the same snapshots.

[fw42]

That seems like an acceptable risk to me... Isn't worth the additional complexity this would add to the codebase, imho. But maybe I'm overestimating this.

[fd0]

I've already thought about having something like a "sync" function that is able to copy data from one repo to another, maybe that's interesting...

[cfcs]

Would be neat to be able to cluster one backup across multiple targets with limited space, but often you want to spend as little time as possible uploading the backup from your primary machines (especially if that's a laptop or a production machine which needs to handle resource load). A capability/signature-aware replication daemon that could do append-only balancing, or "keep newest, purge unneeded objects" operations would be useful.

I believe #256 about ECC could be relevant for the discussion of use cases for this feature.

[fd0]

This feature request could be superseded by #323

[mholt]

@fw42

why in one run? why can't you just backup twice with different destinations?

As fd0 mentioned, it's essentially because I want just one backup in two different places; I want one snapshot, not two. Another reason though, is because I don't want to use all the CPU for two backups when just one will do.

@fd0: If I may, I would like to take a stab at implementing this, if it won't be too difficult. I would need some guidance though. What do you think will be involved?

As I think on this, I wonder if it is even possible to do reliably? Of course it is possible if the multiple destinations/backend repositories are in the exact same state, then it's very easy. But if for some reason one of the repositories is in a different state -- for example, network was down, so only the local destination was backed up to last time -- then I wonder if it is possible to make backing up to the two destinations "at the same time" even worth it. Does that make sense? Restic would be comparing two different sets of diffs, and performing essentially dual backup operations, and combining them into a single invocation just sounds complicated. What do you think?

But if it's possible to get the repositories to be identical before beginning the "hard" or "expensive" part of the backup, maybe then it would be worth it? Can I get your thoughts on this?

[fd0]

I'd like to investigate first if this can be achieved with just rclone. Make a backup to one backend (e.g. local), then synchronize the changes to all the others. Then you'll end up with identical snapshots and data.

When you sync file deletions, too, then you can run forget and prune on your local backend and sync the changes back to the other backends.

It's a two-step process, but it'll keep restic much simpler, and I can't see any downsides...

[mholt]

@fd0 I'd fully support that. I've been thinking about this more and I agree, I don't see why restic should try to do this. If the repos are at all different, they will be effectively two different backups anyway.

As long as rclone can also be used in a proxying configuration similar to restic -> rclone -> backend (and for the second, synchronizing step, it would be rclone -> rclone -> backend then I think we'd be good to go.

I am already doing this manually when I run automated backups to a local destination then sync to the cloud (minus the proxying, but I will need that for my family members!) -- so I know it works. One good question to consider is whether the sync should happen just BEFORE or just AFTER a backup takes place.

If rclone can proxy to another instance of itself and if we're OK with the two-step process, then I'm OK with closing this issue. 👍

Edit: If this goes the rclone route, and I think it will, it might be good for the docs to have an FAQ entry about the recommended way to do this.

[fd0]

Hm. I'm not sure we're talking about the same thing here. My suggestion was:

• Make a backup to local:/srv/backup
• Use rclone (locally) to sync /srv/backup to S3 and B2

When rclone is running somewhere else, it's a bit different:

• Locally run restic backup rclone:/foo, on the other end on a server this runs rclone serve restic /data/storage/repo
• On the server, via cron, call rclone copy /data/storage/repo b2:bucketname:/ and then rclone copy /data/storage/repo s3:otherbucket:/

Then you'll end up with three copies of the repo, one on your server, one on S3, and one on B2.

If you'd like people to be able to access the cloud repositories, give them a way to access an rclone process started with the right remote, like rclone serve restic s3:otherbucket:/.

I'm not sure what you mean with rclone -> rclone -> backend...

Maybe it's helpful to discuss the concrete situation that you'd like to solve in the forum or so.

[mholt]

Sure -- I will post in the forum, although, since this is about rclone, I've posted it in the rclone forum: https://forum.rclone.org/t/proxying-through-a-remote-rclone-instance/5318/1

Basically, my scenario is your first suggestion:

• Make a backup to local:/srv/backup
• Use rclone (locally) to sync /srv/backup to S3 and B2

Except that using rclone locally to sync directly to S3 and B2 requires rclone to have credentials, which in this case the computer doing the backup cannot have access to the credentials. Hence I need a way to rclone to another remote rclone that does have the credentials.

Your second suggestion almost does what I need, except that it involves the server storing the backup on it before it copies to B2 and S3. While that might be desirable for some, I'm trying to avoid an intermediate storage place.

Thanks again! Hope that clarifies some insight into my use case.

[andreasnuesslein]

Hey all,

so I would also very much like the feature of having multiple repositories for one run.

The reasons being:

• scenario "restic backup to backup server and rclone from there": I don't necessarily trust backup location one or two. Either one of them could have a corrupt harddrive, network chipset, a encrypt-all-my-data-on-the-fly-ransomware, ..whatnot.
• scenario "make a local restic backup and then rclone" just simply doesn't work due to limited disk space. Also again the question: what happens if the whole backup gets scrambled by ransomware and I don't realise before I rclone? Will I simultaneously destroy both remote backups?
• lastly: I still would like to tackle the Configuration File #16 issue.
  However due to the reasons stated above and also the idea that a config with multiple repos could just do all the work (instead of me having to create another script that will then chain rclone to restic) I think this here should be done before.

I will totally try to create a PR, but I wanted to make sure it wasn't moot work, so I would like to hear your opinion first @fd0 and others.

cheers :)

[MichaelEischer]

Closing this as the new copy command can be used to sync snapshots between repositories, which also verifies the snapshot integrity while doing so. To clean up snapshots it's e.g. possible to just run forget --keep-something 42 ... for both repositories. Actively using multiple repositories in a single backup run is discussed in #679.

@mholt If this is still relevant, for your use case restic -o rclone.program="ssh remote-host rclone" -r rclone:target-path-passed-to-rclone might work (possibly using a forced command on the server-side).

[Enissay]

Hm. I'm not sure we're talking about the same thing here. My suggestion was:

• Make a backup to local:/srv/backup
• Use rclone (locally) to sync /srv/backup to S3 and B2

When rclone is running somewhere else, it's a bit different:

• Locally run restic backup rclone:/foo, on the other end on a server this runs rclone serve restic /data/storage/repo
• On the server, via cron, call rclone copy /data/storage/repo b2:bucketname:/ and then rclone copy /data/storage/repo s3:otherbucket:/

Then you'll end up with three copies of the repo, one on your server, one on S3, and one on B2.

If you'd like people to be able to access the cloud repositories, give them a way to access an rclone process started with the right remote, like rclone serve restic s3:otherbucket:/.

I'm not sure what you mean with rclone -> rclone -> backend...

Maybe it's helpful to discuss the concrete situation that you'd like to solve in the forum or so.

Is there any recent docker image with cron to trigger a scheduled copy as suggested ?