Validation Pipelines test Scope as a requirement instead of preference

[Trenly]

This issue is related to #147897 and many similar ones which have had issues passing validation due to the dependencies they contain.

Summary

Previously, packages with dependencies would pass validation without issue. At some point in time, the --scope switch was included in the validation pipeline. This has the effect of setting a scope requirement for the WinGet client. Because the upstream dependencies do not always have an installer that can match the requirement specified by the --scope switch, some manifests for packages with dependencies are no longer passing validation.

Requirements -vs- Preferences

The WinGet CLI has two different levels for filtering and selecting which installers can be used. These are requirements and preferences. In general, any filter passed in through command line argument such as --scope, --architecture, or --installer-type is treated as a requirement. The only way to set preferences is through the settings.json file, or by not specifying a preference at all - in which case the CLI has a default preference

Requirements

When a requirement is present, the WinGet CLI will force the requirement to be honored. If either the top-level package or any of its dependencies do not have an installer that meets the requirement, the installation will fail with an error message that no suitable installer was found. This means that if --scope machine is specified, any manifest that either does not specify any scope, or only specifies user scope will cause an install failure.

Preferences

When a preference is present, the WinGet CLI will prioritize installers matching the preference in both the top-level package and its dependencies, but will not fail the installation if the preference cannot be honored. This means that if --scope machine is specified, any manifest with a machine scoped installer will use the machine scope installer, but if a manifest does not specify any scope, or only specifies user scope, the install will use the next best installer (heuristically)

Example

#147897 is a good example of this. The dependency specified in the manifest is Microsoft.VCRedist.2015+.x64. This dependency only has installers available which specify Scope: machine.

Note

The below descriptions assume no user settings for preferences or requirements have been specified in settings.json

If a user attempts to install the manifest from the PR with winget install --manifest <path>, the installation will succeed. This is because there was no scope requirement. The installation should automatically choose the installer for both the top-level package and the dependency, as both are allowed based on user specification.

If a user attempts to install the manifest from the PR with winget install --manifest <path> --scope machine, the installation will succeed. This is because both the top-level package and the dependency have a machine scoped installer. The installation sees the --scope argument as a requirement and forces it to be honored, this will select the machine scoped installer from the PR manifest as well as the appropriate machine scoped installer from the dependency

If a user attempts to install the manifest from the PR with winget install --manifest <path> --scope user, the installation will fail. This is because the dependency does not specify a user scoped installer. The installation sees the --scope argument as a requirement and forces it to be honored, this will select the machine scoped installer from the PR manifest but fails with No suitable installer on the dependency, ultimately cancelling the install.

How this impacts validation

There are a large number of manifests in the repository that do not specify a scope, or only support a single scope. If a new pull request is submitted for a package, and that package has a dependency which does not specify a scope in the manifest, the inclusion of the --scope switch is causing validation to fail. Similarly, if the package has a dependency which does specify scope, then the package will only pass validation if the scope specified in the pull request matches the scope of the dependency.

Using the example above, it can be seen that it most cases, it will install correctly. In the case where the user has specified a requirement that could not be honored, the CLI failed as designed. This brings up an important issue though - How thorough does the validation need to be?

Proposals

1. Use preferences instead of --scope arguments. This would allow the validation pipeline to ensure that if a scope is specified, the installer works for that scope, but doesn't cause failures due to dependencies. The WinGet CLI would still select the appropriate scoped installer from the manifest.
2. Waiver the packages. The packages install correctly in most scenarios, and it is only the edge case in which they are failing - and that failure is by design. I don't believe this is an optimal solution though, as it could result in many packages being waivered, increasing the risk of a package which is truly broken making its way through.
3. Add a switch, user setting, or some way to apply user requirements to the top-level package only, using the requirements as preferences for any dependencies.