Import/export of snapshots

[jo-so]

Output of restic version

restic 0.9.1 compiled with go1.10.2 on linux/amd64

What should restic do differently? Which functionality do you think we should add?

I'm using different storages for backups: an internal repository for daily snapshots and an external disk for not so often snapshots. It would be helpful to transfer a snapshot from the internal repo to the external. So I could run restic -r /mnt/backup export |restic -r /mnt/archive import from time to time and save snapshots on different storages. I think it would be useful to allow options to filter the snapshot, i.e. --exclude …

I'm not sure if both commands should be combined like restic transfer /mnt/backup /mnt/archive, but split command would allow a transfer with ssh and maybe you could add a format option to select the tar format. This would ease the first import of backups from foreign software like restic import --fmt tar /mnt/old/backup.tar. The same for an export as tar.

[jo-so]

This could also help with migrations to newer versions of the repo format.

[mholt]

Very similar to #323 -- I believe the tricky part is to recompute the packs and rewrite the index. Probably doable, but a slight bit expensive (like prune).

(Maybe this issue should be combined with that one.)

[fd0]

Thanks for your suggestion!

It feels to me this this could be achieved with a copy command proposed in #323, although without modifying the snapshots. I'd like to avoid defining (and having to maintain) an export/import format, at least for now. restic's repo format is already rather complex, and maintaining one format is time consuming enough ;)

#323 is blocked at the moment by a config file, apart from the local and sftp backend types it's not possible to define more than one backend e.g. for b2 or s3, so we need to solve that first. Afterwards it's just a matter of traversing the trees in the snapshots and building a list of blobs to copy, then copy the blobs.

So, I'd like to close this issue for now, please subscribe over at #323 to get notified of any (eventual) progress in this area. Please feel free to add further comments, thanks!

[jo-so]

How do you think about a tar import? I had to untar all my old backup and than run restic for them. Wouldn't it be possibly to import a tar archive on the fly?

[rawtaz]

@jo-so Interesting idea. But how often do you need to untar old backups to import them into restic? :)

[fd0]

A tar import has been suggested in #784 and has many other uses, including interesting things such as ssh root@server tar c / | restic backup --read-tar, and with 0.9.0 we've got all the pieces in place. It just needs to be implemented.

[jo-so]

@fd0: What happened to the this idea? Is there any way to copy a snapshot from one archive to another and filter it with commands like bsdtar?

[fd0]

@jo-so nobody implemented this so far. There's PR #2124 which adds exporting snapshots (or parts of snapshots) as a tar.

[Kidswiss]

Also I did propose #2226, which would implement a tar import of sorts. But didn't get much feedback for that unfortunately.