Binary Cache: Add write-back support

[sadroeck]

Write-back support for Binary Caching

This PR adds support for a cache write-back phase during package installation when the relevant cache entry was not newly built. This allows ports to be synced to all binary-cache providers marked as write or readwrite upon a successful installation.

Currently the vcpkg binary-caching layer will only populate a write-capable if the port was built from scratch, i.e. not restored from a read-capable cache provider. This means that a consumer has no way of guaranteeing other caches to be populated, unless they themselves build the port.

Use-cases

As a developer I have a local cache, e.g. file-system & a remote cache, e.g. S3. The remote cache is meant to be a fallback in case the local cache isn't populated. Once restored from the remote cache, I'd like the local cache to be populated with the restored entry, so if the port needs to be restored at a later time, it can be done from the cheaper local cache i.s.o. the remote cache.

Implementation details

Provider identity

The binary caching layer generally works by adding a list of read & write capable cache providers. The read providers are typically single-source, while the write providers are multi-source, e.g. multiple local file-system directories belong to 1 write provider. In order to identify which providers need be written back to, we need to uniquely identify each provider that qualifies for a write-back. This is done using a unique ProviderId, assigned during the parsing of the binary-cache segment list.

Write-back

The push_success() function is now also called post-install to populate the write-capable providers that were (during prefetch) marked as missing the port. These individually mark the cache entry as written back using mark_written_back().

Relevant issues/discussions

• [vcpkg-tool] vcpkg should cache binaries downloaded from AWS locally instead of downloading them every time vcpkg#38684
• [vcpkg] for binary caching give the option to cache locally s3 and others binary sources vcpkg#31500

[sadroeck]

@microsoft-github-policy-service agree company="SingleStore"

[sadroeck]

Added some fixes related to Nuget prefetches & write-only binary providers.