WIP: Smarter cache management for CI. #1652
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This adds a job that runs when a PR is closed (either by being closed or merged into master). This job removes any cache entries owned by the closed PR.
Currently each cache run will create a new copy of the cache files. When a PR is being worked on with many changes happening this can quickly use up the quota of cache space available. With this change old cache files will be removed before the new ones are saved, essentially making each run update the cache instead of storing an updated copy.
|
@m-lindgren : this PR is still marked as a draft - is this correct, or is it ready for review? |
|
@benediktahrens no this is still a WIP, It needs some work and testing. I'm trying to figure out some permission issues currently. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a work in progress trying to make the cache management a bit smarter.
Currently a PR can easily consume all 10 GB available because on each run the cache is duplicated instead of updated. This in and of it self is not a critical problem: GitHub will just evict the oldest cache files until we are below the quota again. It is however a nuisance since it will cause unnecessary rebuilds when a PR is merged, or when another PR is opened.
This does two things so far: