Add options to fine tune VSS snapshots

[DRON-666]

What does this PR change? What problem does it solve?

Original VSS PR use some not-so-universal defaults.

First of all, current Restic architecture don't allow listing all mount points required for backup and we have to enumerate mount points for a given volume and create VSS snapshots for all of them. This can lead to problems if some of this unnecessary mount points links to locked, filled or otherwise unsupported volumes. @fgma has made every effort to avoid this problems, but it's not feasible in all cases. On the other hand, Restic don't follow mount points: if you backup C:\, mount point C:\mnt will be saved as link /c/mnt -> d:\. So snapshotting mount points may only make sense for paths like C:\mnt\somedir.

Extended option -o vss.excludeallmountpoints solves this problem by enterely disabling mount points snapshotting.

Another problem can occurred with backups from multiple volumes (e.g. I backup files with some extensions from all drives), but VSS required only on certain volumes. Why not take a snapshot of them all: because snapshotting of large volumes can take several minutes.

Extended option -o vss.excludevolumes allows you to fine-tune VSS snapshots by excluding specific volumes. Mount points to this volumes are also excluded.

Thus, volumes can be specified in three ways: as drive letter D:\, as mount point C:\mnt, or as volume GUID path \\?\Volume{b151601a-34a4-11e0-b644-806e6f6e6963}\. The trailing slash can always be omitted F:. Before comparison, all specified volumes are converted (with reporting in case of errors) to the form of a volume GUID path, so if C:\mnt links to D:\ then -o vss.excludevolumes=d: and -o vss.excludevolumes=c:\mnt will give the same results.

Also, default VSS provider may lead to some problems. With the option -o vss.provider, vss provider can be specified in three ways: GUID, name and MS as alias for {b5946137-7b9f-4925-af80-51abd60b20d5}.

And the last potential problem is timeout: in my case, it takes more than a minute to take the longest snapshot. It's too close to the default timeout of 120 seconds.

The extended option -o vss.timeout option allows you to change this timeout. My implementation internally uses deadline concept (idiomatic for Go) instead of timeouts.

Was the change discussed in an issue or in the forum before?

I mention some of this issues in the VSS Support PR
Also see timeout issue, provider issue #1 and provider issue #2.

Checklist

• I have read the Contribution Guidelines
• I have enabled maintainer edits for this PR
• I have added tests for all changes in this PR
• I have added documentation for the changes (in the manual)
• There's a new file in changelog/unreleased/ that describes the changes for our users (template here)
• I have run gofmt on the code in all commits
• All commit messages are formatted in the same style as the other commits in the repo
• I'm done, this Pull Request is ready for review

[DRON-666]

Rebased. Fixed some error checking/ignoring.

[jdiercks]

@DRON-666 : Is it too late to ask you to consider one more option to add in this PR?

-o vss.provider="{GUID}"

Backstory: https://forum.restic.net/t/windows-shadow-copy-snapshot-vss-unexpected-provider-error/3674

What do you think? I've done a proof-of-concept hack/patch with a hard-coded GUID for the standard MS system VSS provider, and it solved my problem. I could try rolling my own PR (it would be my first; I have no real experience with Go), but it would seem more expedient to piggy-back on yours, if you're amenable to it.

[DRON-666]

@jdiercks As I expected, the go-ole package is not compatible with all targets.
I don't know how to fix it, but you can compile and use this PR without problems.

[jdiercks]

Tested in my sandbox environment.
Smooth as silk so far.
Hope to roll it into production tomorrow.
Thanks very much!

C:\>restic -r test-repo backup --use-fs-snapshot -o vss.provider=b5946137-7b9f-4925-af80-51abd60b20d5 C:\ N:\ X:\
enter password for repository:
repository 273d5070 opened successfully, password is correct
creating VSS snapshot for [n:\]
use non default vss provider: {B5946137-7B9F-4925-AF80-51ABD60B20D5}
successfully created snapshot for [n:\]
creating VSS snapshot for [x:\]
use non default vss provider: {B5946137-7B9F-4925-AF80-51ABD60B20D5}
successfully created snapshot for [x:\]
creating VSS snapshot for [c:\]
use non default vss provider: {B5946137-7B9F-4925-AF80-51ABD60B20D5}
successfully created snapshot for [c:\]

[fgma]

@jdiercks I'd suggest to use the name of the provider instead of its GUID. GUIDs are a somewhat hard to understand for end users.

[DRON-666]

@fgma Microsoft iSCSI Target VSS Hardware Provider looks no better and, if I remember correctly, can be localized.

[jdiercks]

@fgma I have to agree with @DRON-666 on that one. GUID is more straightforward and reliable for this scenario. If a user is far enough along to be in a situation where explicitly specifying the provider even matters, then they can probably figure out they need to copy and paste a GUID from VSSADMIN LIST PROVIDERS.

[DRON-666]

@fgma @jdiercks Maybe automatic fallback to Microsoft Software Shadow Copy provider 1.0 can resolve all basic provider issues?

[jdiercks]

@DRON-666 that seems to be what VSSADMIN itself does. I would still leave the option in place to override if needed, but fallback would probably save a few people some confusion and grief.

[DRON-666]

that seems to be what VSSADMIN itself does.

@jdiercks I want to check this with API monitor, but I need to find system with more than one provider.
Can you try latest commit without specifying vss.provider option?

[bjoe2k4]

I'm currently using restic in a powershell script that creates a VSS snapshot in the beginning, then calls restic 3 times to backup the same data to 3 different repos (redundancy) and at the end deletes the snapshot. Is there any way this can be simplified with restics own VSS snapshots, but without creating 3 (different) snapshots on every restic call?

[jdiercks]

Can you try latest commit without specifying vss.provider option?

@DRON-666 Looks like without vss.provider it's still using IID_NULL in all the VSS calls.

[DRON-666]

@jdiercks @fgma I just checked: vssadmin always uses VSS_SWPRV_ProviderId not IID_NULL and I would suggest to do the same.
@bjoe2k4 No, you can't do this.

[jdiercks]

@DRON-666 I think I'm fine with restic defaulting to VSS_SWPRV_ProviderId as long as there is an option to override and specify a different provider if needed.

There is a slight risk that changing restic's default behavior would break some other user's backup, if they have multiple providers and they were relying on the hierarchical behavior produced by using IID_NULL, but I suspect that's a corner case at worst, and as long as it's documented they can just start using the new option to get back the results they want.

[jdiercks]

@bjoe2k4 can you share your powershell script? Curious to see how you implemented that. Could be useful to me in a different context.

[bjoe2k4]

@DRON-666 ok thanks, that would have been too good to be true.
@jdiercks unfortunately it is not really polished, so i'm uncomfortable to share. However, for creating snapshots with powershell, have a look at old versions of https://github.com/kmwoley/restic-windows-backup/releases

[DRON-666]

@fgma @jdiercks Now you can use vss.provider={guid}, vss.provider="name" or vss.provider=MS (alias for vss.provider={b5946137-7b9f-4925-af80-51abd60b20d5})

[jdiercks]

@DRON-666 Wow, that's a lot more code! But definitely very flexible. Am I reading right that you decided to keep the original default behavior, with restic passing IID_NULL as provider if none specified via options? I'm not seeing anything that looks like it's doing a fallback to ms sys provider.

[MichaelEischer]

I'm sorry for not taking a look at this PR for several years 😞 ; it turned out to be easier to review than I've expected it to be. Thanks for keeping the PR up to date all this time!

Most comments are just minor nitpicks. The only more important one is that the option names should use kebab case, similar to other already existing options, that is the name should be vss.exclude-volumes instead of vss.excludevolumes and similar for the other option names.

[MichaelEischer]

LGTM. Thanks for addressing the feedback so quick!