Git clone cache #451
Draft
Git clone cache #451
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…anism To work around rate-limit issues, it is now possible for the Client to perform a Git Clone, and to look up certain requests in that cloned version of the repository. The clone is triggered by a request for a list of files (to prevent cloning the entire repository if we only need a single file), and has a default lifetime of 5 minutes. This adds more complexity to the deployment of the client, as it is necessary to do a full clone of the repo, and potentially multiple clones of the repo (if, for example, multiple different revisions are needed) and this requires a potentially large amount of storage. This feature is *disabled* by default, and is only available for GitHub and GitHub Enterprise when enabled.
Also added some basic metrics for cache hits/misses, and exposed /metrics in the Client.
kosti-snyk
previously approved these changes
Nov 8, 2022
kosti-snyk
previously approved these changes
Nov 9, 2022
Also remove retry attempts as I'm not 100% confident they'll work in every situation.
For some reason, apt autoremove will remove git, despite it being installed earlier.
Git creates a very particular SHA, so set that correctly. Also passed back an object rather than a String because too many places would re-Stringify it, which caused problems.
ipsi
force-pushed
the
feat/git-clone-cache
branch
from
db3ca6c
to
b6ae01f
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Add ability for Client to perform a git clone as a caching mechanism
To work around rate-limit issues, it is now possible for the Client to perform a Git Clone, and to look up certain requests in that cloned version of the repository. The clone is triggered by a request for a list of files (to prevent cloning the entire repository if we only need a single file), and has a default lifetime of 5 minutes.
This adds more complexity to the deployment of the client, as it is necessary to do a full clone of the repo, and potentially multiple clones of the repo (if, for example, multiple different revisions are needed) and this requires a potentially large amount of storage.
This feature is disabled by default, and is only available for GitHub and GitHub Enterprise when enabled.
Where should the reviewer start?
How should this be manually tested?
Any background context you want to provide?
Screenshots
Additional questions