Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rdiff-backup replication method #1029

Open
kyrofa opened this issue Dec 15, 2023 · 2 comments
Open

Add rdiff-backup replication method #1029

kyrofa opened this issue Dec 15, 2023 · 2 comments
Labels
enhancement New feature or request

Comments

@kyrofa
Copy link

kyrofa commented Dec 15, 2023

Describe the feature you'd like to have.
The ability to replicate via rdiff-backup. This is our current backup method, but as we move more products to k8s, it has gotten harder to pull off. We started investigating writing an operator to take a snapshot and use rdiff-backup (or perhaps use SnapScheduler in combination with a more simple cron job), but then we discovered VolSync, which is almost a perfect fit except for this lack of support. Instead of building our own, I'd be happy to contribute this support to VolSync, but I wanted to first log an issue and gauge interest.

What is the value to the end user? (why is it a priority?)
rdiff-backup is a reverse incremental backup tool that uses the rsync protocol. It's not really suitable for replication, to be clear, but a way to take space-efficient backups. It generally works over SSH.

How will we know we have a good solution? (acceptance criteria)
A new replication method that looks a bit like the current rsync SSH method, something like the following:

---
apiVersion: volsync.backube/v1alpha1
kind: ReplicationSource
# snip...
spec:
  # snip...
  rdiff-backup:
    sshKeys: volsync-rdiff-dest-sshkeys
    address: my.host.com
    copyMethod: Clone
@kyrofa kyrofa added the enhancement New feature or request label Dec 15, 2023
@tesshuflower
Copy link
Contributor

@kyrofa I've discussed with @JohnStrunk and think we would potentially be interested in this if you're willing to contribute and help maintain the mover. Copying some comments John had from a request for a kopia mover:

By the end of the process, we need to have:

  • Mover code (controller + mover parts)
  • Documentation
  • Tests

Since this will be a fairly substantial piece, starting w/ a design PR is probably a good place to begin.

We'd have a few questions at the start:

  • Do you envision this as a backup/restore operation similar to the way restic works?
    • I.e. a replicationsource that connects via ssh to an existing rdiff server somewhere to backup
    • then a replicationdestination that would restore from that same server?
  • Or is this going to be more of a point-to-point connection like rsync/rsync-tls where you have a source that connects to a destination started by volsync? In this case, do you ever plan on using rdiff restore functionality or would it just be assumed your destination pvc contains the latest files + the extra metadata from rdiff?
    • Additionally if you're looking into this route, I'd recommend you look at rsync-tls as a reference, as the old rsync ssh method will most likely get deprecated soon.

@Neustradamus
Copy link

To follow this ticket

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: No status
Development

No branches or pull requests

3 participants