Replicating binary file difference using VolSync (rsync-tls)

[dheerajjoshim]

Hi
We have a binary file on the volume that gets replicated to the remote site and we are planning to use the VolSync using rsync tls in direct mode
Does rsync only replicate the diff or try to send the entire binary file to the remote site? Because the binary file can grow to 1G and we do not want to send the entire binary file every time the mover pods are created.

[JohnStrunk]

VolSync uses rsync directly to do the replication, so the optimizations are what rsync supports.
A description of the rsync process is here: https://rsync.samba.org/how-rsync-works.html

The short answer is: It should only transfer the changed data blocks within the file, but the full file will be read (on both sides) in order to calculate the differences.

[dheerajjoshim]

Thanks.
What I understood is, that if the checksum of the block is changed then only that data is sent. If for any reason checksum of every block changes then it will end up sending the entire file.

Did VolSync test and publish any performance results for such large binary file replication? Of course under test scenarios where binary file changes in a predictable manner?

[JohnStrunk]

That's correct.

We have not published performance numbers, primarily because we don't have a good way of standardizing the environment such that someone could get useful information from the data.
The performance is affected by the network connection (BW and latency), the individual nodes' CPU and memory, the underlying file system performance (both data and metadata performance), etc.
There's really no substitute for trying it out in your own environment.

[dheerajjoshim]

Thanks.
Of course, we will test it in our environment.
But, we were looking for some reference performance numbers for a given setup and condition.