Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add bounded concurrency for tag lookup and untag #4329

Merged
merged 2 commits into from Apr 26, 2024
Merged

add bounded concurrency for tag lookup and untag #4329

merged 2 commits into from Apr 26, 2024

Commits on Apr 26, 2024

  1. vendor: update manifest dependencies 

    Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
    @microyahoo
    microyahoo committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    a5882d6 View commit details
    Browse the repository at this point in the history

  2. add concurrency limits for tag lookup and untag 

    Harbor is using the distribution for it's (harbor-registry) registry component.
The harbor GC will call into the registry to delete the manifest, which in turn
then does a lookup for all tags that reference the deleted manifest.
To find the tag references, the registry will iterate every tag in the repository
and read it's link file to check if it matches the deleted manifest (i.e. to see
if uses the same sha256 digest). So, the more tags in repository, the worse the
performance will be (as there will be more s3 API calls occurring for the tag
directory lookups and tag file reads).

Therefore, we can use concurrent lookup and untag to optimize performance as described in goharbor/harbor#12948.

P.S. This optimization was originally contributed by @Antiarchitect, now I would like to take it over.
Thanks @Antiarchitect's efforts with PR distribution#3890.

Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
    @microyahoo
    microyahoo committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    a2afe23 View commit details
    Browse the repository at this point in the history