Error building windows container inside a windows container; failed to prepare: failed to create scratch layer: no parent layers present: unknown #4826
Comments
|
fwiw, i also tried this in a HostProcess pod and got the exact same result, which is somewhat surprising since that should be equivalent to running buildkitd.exe directly on the host (edit) also same issue directly on the host, so i am now not sure this ticket has anything to do with the extra container layer :) |
|
@bnu0 -- thanks for checking this out! I'm going to take a look at this. In the meantime, please lmk the buildkit version you are using, will assume the latest. |
|
Oh, wait, if I get you correctly, you are trying to build a Windows container inside a Windows container? That is not supported currently, from the Windows platform side; by design. |
|
It didn't, same error. After filing this issue I tried the HostProcess pod, same issue, and then tried buildctl+buildkitd directly on the kubelet host which also failed with the same error. So I think perhaps this is some issue related to windows 2019, or to sharing a containerd layer cache between k8s (kubelet via cri) and buildkitd? |
|
And yes I was ideally trying to build a container from inside another - but HostProcess would be okay if I could get it to work too, my goal is really |
This is a reasonable ask and one we will prioritize. I know that nested containers is something on the backlog of the Windows Containers platform team. I don't have any timelines for now but I'll keep you posted here as soon as I get something concrete. |
I will repro the scenario with WS2022 and report my findings. |
|
Hey, and on the buildkitd side: |
you need to use containerd v1.7.7 or later |
|
Gonna give this a shot and report back! |
|
@bnu0 wondering if you ever tried this? I actually tried this before i seen this issue and I was coming up with |
|
Well, I got a bit further, I was able to use a HostProcess to be able to run buildkitd and buildctl. I'm looking to run buildkitd on a daemonset of pods,and I can get it to run, but it seems that I get an access denied when trying to write to C:/Windows/SystemTemp Curious if you got a similar behavior? |
|
yes, you can't run these things isolated. they will have privileges. you could experiment with a hyperv container but it's unlikely to work. for the foreseeable future, on windows, building a container that uses |
|
I did! I was able to have |
Hi! First of all, thanks to the team and to @profnandaa for driving this effort! We seem so close to having a robust windows container ecosystem, and oci builds are the last big missing piece. very exciting!
We're trying to move our CI builds of windows containers themselves into Kubernetes, and specifically into other windows containers. I'm not sure this is a supported/possible solution, although we seem to be pretty close to having it working, but we're getting stuck somewhere near buildkit's
cacheManager:Background
On the hosts, we're using EKS-optimized Full windows images, with containerd v1.6.18. Containerd on the host is listening on the standard named pipe at
\\.\pipe\containerd-containerd.We're running pods which match the following spec, note that we are mounting in the containerd grpc pipe, and also the root of containerd's data folder (although i tried both with and without this):
We can shell into the pod; once inside,
ctrworks and can list and run containers on the host.runonly works with--detachthough - seemingly because the stdio pipes are created on the host:Invocation
We make the simplest possible
Dockerfileinside the container:When we invoke
buildctl buildwe get the above failure:In addition to the same thread dump as above, the
buildkitd.exelogs contain:Debugging info:
I'd like to understand if this solution is simply impossible on windows (and we must use HostProcess pods) or if i can get around it with configuration. Thanks!
The text was updated successfully, but these errors were encountered: