Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Syncoid only uses its own bookmarks #588

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Syncoid only uses its own bookmarks #588

wants to merge 1 commit into from

Conversation

durkatlon
Copy link

Currently syncoid will use any bookmark it can find for a sync request. When setting a bookmark it does not leave behind an identifying mark. It never removes bookmarks.

With these changes, bookmarks are handled much like the sync snapshots that syncoid sets (and cleans up when it's done with them).

  • A bookmark that is set has syncoid${hostname} on the end of the name.
  • Syncoid will only look for its own bookmarks when deciding which one to use.
  • When it's done using a bookmark, it is cleaned up.
  • Also handles bookmark to oldest snap then to newest snap.

I'm using this locally to create a 3rd backup of a 2nd-ary backup that is populated by pve-zsync (ProxMox) which uses zfs send/receive and would thereby wipe out the syncoid sync snaps from the tertiary backup box.

@durkatlon
Syncoid only uses its own bookmarks
8a39d39 
 - Filter bookmarks to end in _syncoid_hostname
 - When using a bookmark, clean it up

Makes syncoid bookmarks behave like sync snaps.
phreaker0
phreaker0 requested changes Nov 3, 2020
View reviewed changes
Copy link
Collaborator

@phreaker0 phreaker0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, I implemented the bookmark stuff and it's designed to use any available bookmark to guarantee a working replication.

@durkatlon
Copy link
Author

Hi, I implemented the bookmark stuff and it's designed to use any available bookmark to guarantee a working replication.

How about if I changed it like this:

  • Bookmarks set by syncoid itself will still follow my new naming convention.
  • For replication it will preferentially look for a bookmark set by syncoid from the current host. However it will use any available bookmark, if such a bookmark cannot be found.
  • If a syncoid bookmark belonging to the host that the script is running on is used as the source of a replication step, it subsequently gets deleted.
  • If a non-syncoid bookmark is used, it does not get deleted.
  • If a syncoid bookmark belonging to a different host is used, it does not get deleted.

This would still satisfy my main requirement of not leaving behind an ever-growing stack of bookmarks, and it should still satisfy your requirement that any available bookmark is used to perform the sync.

What do you think?

@0xFelix
Copy link
Contributor

0xFelix commented Feb 20, 2021

I'm backing up irregularly to multiple host and your proposal would fit my use case perfectly, too.

Currently backing up irregularly to more than two destinations prints a scary warning (CRITICAL ERROR) that a bookmark could not be created because a bookmark without suffix and a bookmark with a guid suffix already exist. The bookmarks identify the exact same source snapshot and so no harm is done and the replication succeeds in spite of that. Being able to identify bookmarks could solve this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@phreaker0 phreaker0 phreaker0 requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@durkatlon @0xFelix @phreaker0