Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buildx build --platform ... uses wrong image #844

Open
andyli opened this issue Nov 17, 2021 · 9 comments · May be fixed by moby/buildkit#4924
Open

buildx build --platform ... uses wrong image #844

andyli opened this issue Nov 17, 2021 · 9 comments · May be fixed by moby/buildkit#4924

Comments

@andyli
Copy link

andyli commented Nov 17, 2021

When building FROM an image not built for the target platform, buildx just uses it without error, but I think it should throw.

For example,

FROM mcr.microsoft.com/vscode/devcontainers/base:0-focal
RUN uname -m

Building for linux/arm64:

$ docker buildx build --no-cache --progress=plain --platform linux/arm64 .
WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load 
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 126B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for mcr.microsoft.com/vscode/devcontainers/base:0-focal
#3 DONE 0.4s

#4 [1/2] FROM mcr.microsoft.com/vscode/devcontainers/base:0-focal@sha256:16c98716ca86b6159814442e8d752d7df6d7c6d378a85c4d5a008b8ef5058686
#4 resolve mcr.microsoft.com/vscode/devcontainers/base:0-focal@sha256:16c98716ca86b6159814442e8d752d7df6d7c6d378a85c4d5a008b8ef5058686 done
#4 CACHED

#5 [2/2] RUN uname -m
#5 0.069 x86_64
#5 DONE 0.1s

Notice uname -m printed x86_64.

My buildx version:

$ docker buildx version
github.com/docker/buildx 0.6.3+azure 266c0eac611d64fcc0c72d80206aa364e826758d
@tonistiigi
Copy link
Member

tonistiigi commented Nov 17, 2021

If mcr.microsoft.com/vscode/devcontainers/base:0-focal is a manifest list(defines its platform in the manifest) then it will throw an error. On single platform images, it does not for backward compatibility.

@andyli
Copy link
Author

andyli commented Nov 18, 2021

Is there a case in which getting an image of not the specified --platform can be useful? If so, it may be better have this behavior as an opt in (e.g. introduce a --allow-platform-fallback) instead of the default.

At least buildx should show a warning.

@kamermans
Copy link

I agree with @andyli. Unless I'm missing something, as it stands today, the only way to prevent an automated build system using buildx from happily building and pushing images for the wrong platform is to parse the Dockerfile and check each FROM statement with a complicated script that calls the Docker registry API to figure out if the base image is a v2/multi-arch image or not (which is what I'm doing). This is extremely disconcerting and frankly, dangerous. Even an opt-in flag like --require-platform or even --min-manifest-version=2 or --verify-platform or something would be a big improvement.

@tonistiigi
Copy link
Member

@thaJeztah @crazy-max I think we have given people enough time to move away from broken builders where the config and binary arch does not match and upgrade to multi-arch. I'm ok with breaking backward compatibility and making this case error. We could provide a build-arg to bypass the error. WDYT?

@plinehan
Copy link

Just want to point out another really confusing thing here:

If you run a docker image inspect on the newly-built image, it shows "Architecture": "arm64" even though it is amd64.

This may be for some sort of backward compatibility (?) but it was a big red herring for me when I was investigating.

@andreisilviudragnea
Copy link

@tonistiigi Any update on this issue? I came across this behaviour and it took me by surprise too.

@polarathene
Copy link

I recently split a Dockerfile into smaller files and one of the build stages used FROM --platform= for a stage to use a different RUN instruction based on the platform as different packages were needed to install / build.

When using docker buildx bake to build the sequence of files and leverage context to reference earlier stages from prior Dockerfiles, I noticed that regardless of where I specified explicitly the linux/amd64 platform, it was always running the 2nd stage in a Dockerfile for the linux/arm64 platform.

Just adding another scenario with local image builds where this was a surprise as I didn't want the ARM64 stage as part of the AMD64 build, thinking FROM --platform would have been respected.

@tonistiigi
Copy link
Member

@polarathene Create a separate issue with a reproducer. It is unlikely that this is the same issue if you are combining builds with bake using named contexts.

@argh
Copy link

argh commented May 26, 2024

Count me as yet another person that found this behavior beyond surprising. We were trying to build a multi-arch image using --platform linux/arm64,linux/amd64. We do have some legacy images where there is no arm64 base, and my expectation was that either it would build just for x86 and give warnings for arm, or else it would generate an error for the whole build. I could have worked with either behavior.

Instead, it "successfully" built two images, apparently multi-arch when checked from the outside. Running docker buildx imagetools inspect shows both architectures. Everything in the build process looked successful. But then doing a docker run --platform linux/arm64 and docker run --platform linux/amd64 both show that they're actually both x86. (The fact that the run command didn't complain that the architecture mismatched was also surprising and distressing!)

Fallbacks are always risky without some compensating control to allow strictness to be enforced, but generating something totally contrary to the inputs while simultaneously claiming that they do, and not generating a single warning... this is a critical bug in my view. It's hard to see how it's been open since 2021.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants