use_embedded_backup_restore & compression & differential backups

[artiommocrenco]

Hello,

I am using use_embedded_backup_restore: true

Please suggest if I can use compression (or is it already compressed?). Because here's what I am getting

error `general->remote_storage: sftp` `clickhouse->use_embedded_backup_restore: true` require sftp->compression_format: none, actual bzip2 logger=validateUploadParams

Also, I can not understand why my incremental backups take almost the same size as full backups. I upload them using --diff-from but still when I do clickhouse-backup list remote, they seem to be the same size as full backups. I tested by creating two backups and trying to upload one as full backup and the other as incremental.

[Slach]

clickhouse already store data in compressed format

for use_embedded_backup_restore: true
data already uploaded to backup disk, and you need upload only metadata, so doesn't need to more compression

support incremental backup with use_embedded_backup_restore: true will released in 2.5.0

--diff-from have no sense for use_embedded_backup_restore: true
better use create_remote --diff-from-remote

[Slach]

look to #843 and watch progress about incremental backup for embedded

[artiommocrenco]

@Slach thanks a lot

so, --diff-from does not work with use_embedded_backup_restore: true, while --diff-from-remote does?

If I understand it correctly, I can use --diff-from-remote with both create_remote or upload

[artiommocrenco]

@Slach I am asking because you are suggesting to use --diff-from-remote while at the same time you are saying that support for incremental backups is not there yet (will be released)

so, I am wondering if specifying --diff-from-remote has any effect then

[Slach]

--diff-from-remote with embedded will work when 2.5.0 will released
i suggest use it when
--diff-from have no sense and will not use with embedded

[artiommocrenco]

--diff-from-remote with embedded will work when 2.5.0 will released

i suggest use it when

--diff-from have no sense and will not use with embedded

Thank you very much for help!