Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

version-checker seemingly leaks memory and gets oom-killed #76

Open
roobre opened this issue Feb 20, 2021 · 2 comments
Open

version-checker seemingly leaks memory and gets oom-killed #76

roobre opened this issue Feb 20, 2021 · 2 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@roobre
Copy link

roobre commented Feb 20, 2021

I am running version-checker on a single node, quite small cluster with ~60 pods. So far it is working nicely, but I do not understand the memory behavior it has.

I'm basically running the sample deployment file, plus the --test-all-containers flag and some cpu limits:

        resources:
          requests:
            cpu: 10m
            memory: 32M
          limits:
            cpu: 50m
            memory: 128M

kubectl get pod -o yaml

Over time, I see that version-checker approaches the memory limit and then stays near ~99% for a while. After some time, the kernel kills the ct due to OOM and k8s restarts the pod.

Memory chart

However, I do not see anything alarming in the logs, other than some failures and expected permission errors.

This doesn't seem to have any functional impact, but does fire some alerts and doesn't look good on my dashboards :)

Is this behavior intended, and/or is there any way to prevent it?

@trastle
Copy link

trastle commented Feb 24, 2021

We are seeing a similar behaviour while running Version Checker. Would be interested to know if there are recommended values for the limits?

@Trede1983
Copy link

Also seeing something similar with Version Checker getting OOM killed fairly frequently.

@davidcollom davidcollom added enhancement New feature or request help wanted Extra attention is needed labels Jul 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

4 participants