Changing compression level does not seem to reduce the overall repo size

[televisi]

Hello,

So, I just started to use borg.
According to the FAQ, we can change compression level/algorithm anytime and it will not affect the deduplication.

Question:
Does it mean, the total repo size won't significantly reduced even after the second/consecutive archives runs with higher compression level?
It will only significantly reduced if you start the repo from start again?

Scenario:

1. Initial archive, I use zstd compression (standard/ modern wide-range compression)
2. Soon after that, I run a second archive (let's say only a small number of files difference - in KBytes), with the lzma,9 compression (Low speed, high compression)

Let's say I decided to delete the first archive and run borg compact; will it reduce the TOTAL REPOSITORY SIZE? consider the second archive has a higher level of compression?

Real Scenario:
I have two archives:

1. Archive 01 = documents-2024-02-04_T_09:24:02 => use zstd compression
2. Archive 02 = documents-2024-02-04_T_16:24:02 => use lzma,9 compression

When I ran Vorta UI app, it shows the second archive is only 0.6MB size:

While, from the command line:
borg info ssh://user@repo_url/./repo::documents-2024-02-04_T_09:24:02

This archive:              119.10 GB             70.15 GB              1.40 MB
All archives:              238.19 GB            140.30 GB             53.21 GB

                       Unique chunks         Total chunks
Chunk index:                   91127               343478

borg info ssh://user@repo_url/./repo::documents-2024-02-04_T_16:24:02

Utilization of maximum supported archive size: 0%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:              119.09 GB             70.15 GB            637.14 kB
All archives:              238.19 GB            140.30 GB             53.21 GB

                       Unique chunks         Total chunks
Chunk index:                   91127               343478

Thanks

[ThomasWaldmann]

The complete faq entry:

Can I safely change the compression level or algorithm?

The compression level and algorithm don’t affect deduplication. Chunk ID hashes are calculated before compression. New compression settings will only be applied to new chunks, not existing chunks. So it’s safe to change them.

[ThomasWaldmann]

The reason for that faq entry is that some people thought the chunk ID hash is computed after compression and thus changing the compression algorithm would change the ID - which is not the case because borg computes the ID before compression.

[ThomasWaldmann]

To answer your question: yes, because the 2nd archive references most chunks that were produced by the 1st archive creation. Only the different, new chunks were compressed using the new algorithm.

If you want to globally recompress, borg 1.x only has borg recreate, but it is rather slow / inefficient.

borg2 will have a more efficient way to globally recompress.