Replies: 2 comments 4 replies
-
|
Having the same problem: yes and no (usually no). Correct is that borg is currently single-threaded and your first backup using borg will take a relatively long time. But (and this is different from older backup tools), due to the chunk-based deduplication, all following backups of the same data sets will be much much faster, because it will not process/store data chunks again which it already has in the repository (== backup storage). Additionally, borg uses some clever tricks with the "files cache" and in-memory hash tables, always making full backups super fast (if a lot of files did not change at all, which is often the case). Thus, while multithreading is on the roadmap, it is not as important and not as pressing as one might think it is. It's often just a "one-time patience issue" (initial backup) and avoiding CPU work is always better than parallelising cpu work. Of course, multithreading becomes increasingly more important with growing amounts of changed chunks between backups. Folks who produce lots of new data all the time need it. Thus, next releases (2.0, 2.1) will address mostly improving crypto and getting rid of old code and some other long-term open issues that we could not yet fix without breaking compatibility. The better crypto will also unblock/solve some multi-repo / multi-client issues of the past. Once we're finished with that, guess it will be the time for adding multithreading. |
Beta Was this translation helpful? Give feedback.
-
|
Ok. Interesting. Thanks for your comment on the subject. I shouldn't be saying this, but I look forward to getting back from holidays and trying my first backup 🙂 |
Beta Was this translation helpful? Give feedback.
-
|
Just a follow up on this. I got back from holidays and did a backup using lz4 compression and it was actually very fast. I didn't even see it hit 100% of the CPU. Single threaded performance was good. It took me roughly 1hr to backup 650GB. I think that's roughly 185MB/s. Quite good. |
Beta Was this translation helpful? Give feedback.
-
|
How many files per second is that? Maybe use |
Beta Was this translation helpful? Give feedback.
-
|
1074727 files total
I ran cancelled and resumed the backup a few time so I'm not sure exactly
how long it took.
Does the final printout of stats give total runtime or the time of the most
recent backup run?
from my mobile
…On Wed, 24 Aug 2022, 7:59 pm TW, ***@***.***> wrote:
How many files per second is that? Maybe use --stats --info --list to
check.
—
Reply to this email directly, view it on GitHub
<#6958 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOIXVLHC22HR7EU3XRDNYLV2YFCRANCNFSM56AQPPSQ>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
It refers to the current run and archiving start/end refer to the time needed for the actual backup (not including startup/shutdown overhead of borg). |
Beta Was this translation helpful? Give feedback.
-
I'm currently on holiday and only have my tablet, but I'm looking to migrate away from Deja Dup / Duplicity due to poor backup performance. It runs on one CPU core only and takes many many hours to do a backup. Based on what I've seen in the docs and the issues on GitHub, Borg will have the same problem.
It seems like this is something that's on the roadmap though so I'm interested in learning what the current state of affairs is. Is this planned for any particular version in the future? Any and all info appreciated
P.s. Borg looks great from what I've read. Really hope I can send up using it
Beta Was this translation helpful? Give feedback.
All reactions