Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Staggered File Versioning - intervals as parameters #8352

Open
mziehm opened this issue May 21, 2022 · 0 comments · May be fixed by #9094
Open

Staggered File Versioning - intervals as parameters #8352

mziehm opened this issue May 21, 2022 · 0 comments · May be fixed by #9094
Labels
enhancement New features or improvements of some kind, as opposed to a problem (bug)

Comments

@mziehm
Copy link

mziehm commented May 21, 2022

I'm concerned about the maximum storage need of Staggered File Versioning. If I understand correctly, for the first 1 hour alone a maximum um 120 copies might be kept, and for the first day 24 additional, meaning with only 1 day Maximum Age, one would need to ensure the 144 fold storage space of the synced folder to be save. Otherwise an adversary (human or software e.g. ransomware) can prevent any syncing by flooding the versioned storage.
While I do not doubt that there are use cases where the current setup is ideal and either backed by sufficient storage or the risk is acceptable, I would like to suggest and ask for making the staggered intervals configurable, which would be also in general a welcome increase in flexibility with very limited additional code complexity.

fields to allow configuring interval (i.e. parameterize line 50 to 53 of staggered.go (https://github.com/syncthing/syncthing/blob/main/lib/versioner/staggered.go)

			{30, 60 * 60},                     // first hour -> 30 sec between versions
			{60 * 60, 24 * 60 * 60},           // next day -> 1 h between versions
			{24 * 60 * 60, 30 * 24 * 60 * 60}, // next 30 days -> 1 day between versions
			{7 * 24 * 60 * 60, maxAge},        // next year -> 1 week between versions

(Users can tune the staggering to their usecase, and syncing needs resulting in e.g.
{15 * 60, 2 * 60 * 60}, //first 2 hours -> 15min between version
{2 * 60 * 60, 24 * 60 * 60} //next day -> 2h between versions,
...

So one would need a input table with two columns and fixed or variable number of rows:
bin | duration
30 | 3600
3600 | 86400
86400 | 2592000
604800 | maxAge
would be the current setup (and then default)

Advantage:

  • parameterization should not be to hard and no other changes to the code or test needed
  • adjustable usecases, storage and risk settings

This suggestion would also solve #4717

Please consider and many thanks for this great piece of software
@mziehm mziehm added enhancement New features or improvements of some kind, as opposed to a problem (bug) needs-triage New issues needed to be validated labels May 21, 2022
@AudriusButkevicius AudriusButkevicius removed the needs-triage New issues needed to be validated label Sep 11, 2022
tomasz1986 added a commit to tomasz1986/syncthing that referenced this issue Sep 10, 2023
@tomasz1986
gui, lib/versioner: Allow to configure staggered versioning intervals (…
9dfe23f 
…fixes syncthing#8352)

Currently, there are four intervals used in the staggered versioning,
with all of them but the Maximum Age being hard-coded. With this change,
the user is able to configure all intervals to their liking. Due to the
complexity of the configuration, this option is intended to be used by
advanced users only.

In addition to the above, also:

1. Add yet another interval for more fine-grained configuration
   possibilities. By default, the new interval activates after one year,
   keeping one version per month. However, because the default Maximum
   Age is also set to one year, this fifth interval is actually not
   utilised unless the user increases the Maximum Age to a higher value.

2. Fix a small bug in the Staggered Versioning test where the default
   time period for the third interval was supposed to be set to 30 days,
   but in reality it was set to just 7 days.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
tomasz1986 added a commit to tomasz1986/syncthing that referenced this issue Sep 10, 2023
@tomasz1986
gui, lib/versioner: Allow to configure staggered versioning intervals (…
1003d67 
…fixes syncthing#8352)

Currently, there are four intervals used in the staggered versioning,
with all of them but the Maximum Age being hard-coded. With this change,
the user is able to configure all intervals to their liking. Due to the
complexity of the configuration, this option is intended to be used by
advanced users only.

In addition to the above, also:

1. Add yet another interval for more fine-grained configuration
   possibilities. By default, the new interval activates after one year,
   keeping one version per month. However, because the default Maximum
   Age is also set to one year, this fifth interval is actually not
   utilised unless the user increases the Maximum Age to a higher value.

2. Fix a small bug in the Staggered Versioning test where the default
   time period for the third interval was supposed to be set to 30 days,
   but in reality it was set to just 7 days.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
@tomasz1986 tomasz1986 linked a pull request Sep 10, 2023 that will close this issue
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New features or improvements of some kind, as opposed to a problem (bug)
Projects
None yet
Milestone
No milestone
2 participants
@AudriusButkevicius @mziehm