Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why kind load docker-image <docker_image_name> would load to control node? #3465

Open
xihajun opened this issue Dec 28, 2023 · 1 comment
Open
Labels
kind/support Categorizes issue or PR as a support question.

Comments

@xihajun
Copy link

xihajun commented Dec 28, 2023

kind load docker-image xxx:latest -n testgpu

I guess we don't need to load to control-plane? Image: "xxx" with ID "sha256:747d6cf14415fb5f48f8dbec4ba28dcb4b8339347540e12e57aac3438487a738" not yet present on node "testgpu-control-plane", loading...

Image: "xxx" with ID "sha256:747d6cf14415fb5f48f8dbec4ba28dcb4b8339347540e12e57aac3438487a738" not yet present on node "testgpu-control-plane", loading...
Image: "xxx" with ID "sha256:747d6cf14415fb5f48f8dbec4ba28dcb4b8339347540e12e57aac3438487a738" not yet present on node "testgpu-worker", loading...

Details:

Image: "xxx" with ID "sha256:747d6cf14415fb5f48f8dbec4ba28dcb4b8339347540e12e57aac3438487a738" not yet present on node "testgpu-control-plane", loading...
Image: "xxx" with ID "sha256:747d6cf14415fb5f48f8dbec4ba28dcb4b8339347540e12e57aac3438487a738" not yet present on node "testgpu-worker", loading...
ERROR: failed to load image: command "docker exec --privileged -i testgpu-control-plane ctr --namespace=k8s.io images import --all-platforms --digests --snapshotter=overlayfs -" failed with error: exit status 1
Command Output: unpacking docker.io/library/xxx:latest (sha256:b1834fcf8afb3d81cd58d6c7058b33f5ad1d04d8d9f9fd8ca5040ed6271a2b7f)...time="2023-12-28T16:58:16Z" level=info msg="apply failure, attempting cleanup" error="failed to extract layer sha256:b7685fd66665733a9b43bf7f3052b590af89af4eccdb69d4bd40887fe5044b9f: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/109/fs/usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cuda.so: no space left on device: unknown" key="extract-44196102-klKV sha256:aa10e0c10b2bdf3648d57d23b7012afc73d583005aa50b1218c11b643a9ef047"
time="2023-12-28T16:58:16Z" level=warning msg="extraction snapshot removal failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device: unknown" key="extract-44196102-klKV sha256:aa10e0c10b2bdf3648d57d23b7012afc73d583005aa50b1218c11b643a9ef047"
ctr: failed to extract layer sha256:b7685fd66665733a9b43bf7f3052b590af89af4eccdb69d4bd40887fe5044b9f: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/109/fs/usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cuda.so: no space left on device: unknown

@xihajun xihajun added the kind/support Categorizes issue or PR as a support question. label Dec 28, 2023
@BenTheElder
Copy link
Member

You could schedule workloads to those nodes as long as you set tolerations (which may be the case for e.g. https://kind.sigs.k8s.io/docs/user/ingress/ or other system components), you can set which nodes you load to using the flags.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/support Categorizes issue or PR as a support question.
Projects
None yet
Development

No branches or pull requests

2 participants