Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Managed CRI docker image cache #567

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
Draft

Conversation

koct9i
Copy link
Collaborator

@koct9i koct9i commented May 2, 2024

  • Add weight for cookies in async SLRU cache
    Weighted cookie allows to reclaim space before inserting new value.
    This helps to avoid temporary overcommit.

    Cookie weight don't have to exactly match weight of final value,
    and could be updated during insertion.

  • Add async SLRU method IsMustRetain to detect and preserve useful entries

  • Always retry chained docker image pull
    Without unused "error" argument retry does not happens.

    Fixes: c437f3b ("Cosmetics")

  • Add CRI Image Cache
    This is SLRU cache which tracks docker images and their tags and digests.

@koct9i koct9i requested a review from gritukan May 2, 2024 13:46
yt/yt/library/containers/cri/config.cpp Outdated Show resolved Hide resolved
yt/yt/library/containers/cri/config.h Outdated Show resolved Hide resolved
yt/yt/library/containers/cri/image_cache.cpp Outdated Show resolved Hide resolved
yt/yt/library/containers/cri/image_cache.cpp Outdated Show resolved Hide resolved
yt/yt/library/containers/cri/image_cache.cpp Outdated Show resolved Hide resolved
yt/yt/library/containers/cri/image_cache.cpp Outdated Show resolved Hide resolved
yt/yt/library/containers/cri/image_cache.cpp Outdated Show resolved Hide resolved
yt/yt/library/containers/cri/image_cache.cpp Outdated Show resolved Hide resolved
yt/yt/library/containers/cri/image_cache.cpp Outdated Show resolved Hide resolved
yt/yt/library/containers/cri/image_cache.cpp Outdated Show resolved Hide resolved
@Kontakter Kontakter requested a review from k-pogorelov May 3, 2024 14:35
@koct9i koct9i force-pushed the cri-image-cache branch 2 times, most recently from 7cb135a to e9707da Compare May 6, 2024 16:09
Weighted cookie allows to reclaim space before inserting new value.
This helps to avoid temporary overcommit.

Cookie weight don't have to exactly match weight of final value,
and could be updated during insertion.
This helps to avoid overcommit by pinned entries which could live arbitrary
long time after removing from cache.

Also this provides better estimate for end of "last recent use" because
"touch" happens at the beginning of use not at at the end.
Without unused "error" argument retry does not happens.

Fixes: c437f3b ("Cosmetics")
This is SLRU cache which tracks docker images and their tags and digests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants