Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail on missing clone #817

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

olidal
Copy link

@olidal olidal commented Apr 21, 2023

Adds new option --no-full-clone to syncoid:

--no-full-clone
Only applies to recursive transfer when clone handling is active.
In case a clone origin is missing (on destination) because its transfer was interrupted don't switch to a full clone sync.
Example worst case scenario this may help prevent:
A 1TB dataset has 500 identical clones. Overall it uses just 1 TB storage space at the source, because all clone share the storage with origin.
Assume a recursive transfer is started that needs to send all datasets (origin and 500 clones).
Assume also the origin dataset transfer ends up failing. Without selecting this option, the recursive transfer proceeds to next recursive dataset, and because origin is missing (receive not completed), each clone becomes a full clone and we end up trying to transfer 500TB !

@phreaker0
Copy link
Collaborator

it would be nice to have an improved clone handling like #572 instead of a workaround like this, but because this is very simple we can add it for now until we have proper clone handling with dependency handling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants