packaging: refactor docker image build #2566
Conversation
| VERSION ?= $(shell git rev-parse HEAD) | ||
| REGISTRY ?= registry-write.opensource.zalan.do/teapot | ||
| BINARIES ?= skipper webhook eskip routesrv | ||
| IMAGE ?= $(REGISTRY)/skipper:$(VERSION) | ||
| ARM64_IMAGE ?= $(REGISTRY)/skipper-arm64:$(VERSION) | ||
| ARM_IMAGE ?= $(REGISTRY)/skipper-armv7:$(VERSION) | ||
| PACKAGE ?= github.com/zalando/skipper | ||
| MULTIARCH_IMAGE ?= container-registry-test.zalando.net/teapot/skipper:$(VERSION) |
There was a problem hiding this comment.
Added this for consistency with other *_IMAGE variables.
AlexanderYastrebov
force-pushed
the
packaging/refactor-docker-build
branch
2 times, most recently
from
6924001
to
e7ea94b
Compare
|
👍 |
| @@ -1,16 +1,18 @@ | |||
| ARG BASE_IMAGE=default | |||
| FROM registry.opensource.zalan.do/library/alpine-3:latest AS default | |||
| FROM alpine:latest AS default | |||
There was a problem hiding this comment.
I thought it would be more consistent:
- we override multiarch base image via build arg
- this Docker file by itself now depends on the stock alpine
- otherwise we would need to override base image for arm64 and armv7 (non-multiarch)
There was a problem hiding this comment.
If you switch the base image from a single-arch to a multi-arch it should just work (if the requested arch is supported, but alpine supports all we need). So you should be able to use alpine:latest for "arm64 and armv7 (non-multiarch)" as well. (or the equivalent Zalando variant container-registry.zalando.net/library/alpine-3:latest, note that this is not a public image)
| .PHONY: docker.build.amd64 | ||
| docker.build.amd64: clean build.linux.amd64 docker.build.enable ## build linux/amd64 image using a trusted amd64 alpine base image | ||
| docker buildx build -t $(IMAGE) --platform linux/amd64 --load . \ | ||
| --build-arg BASE_IMAGE=registry.opensource.zalan.do/library/alpine-3:latest |
There was a problem hiding this comment.
Why use a custom base image here? Is it for compliance with Zalando? The default alpine:latest would also work.
There was a problem hiding this comment.
container-registry.zalando.net/library/alpine-3:latest would also work as it's just the compliant version of alpine:latest (more correctly alpine:3).
There was a problem hiding this comment.
Is it for compliance with Zalando?
Yes, so the idea is that Docker file uses opensource image and Makefile configured base image we need for us
There was a problem hiding this comment.
Then best use alpine:latest (or with a version) by default for open source and overwrite it in Makefile with container-registry.zalando.net/library/alpine-3:latest (for us).
There was a problem hiding this comment.
Latest will break OpenSSF Scorecard because they want sha based image versions. I think it's also better being explicit of versions and have automations that change it. In the future we can likely merge these pr automatically as discussed in an internal meeting.
| docker buildx build --rm -t $(MULTIARCH_IMAGE) --platform linux/amd64,linux/arm64 --push \ | ||
| --build-arg BASE_IMAGE=container-registry.zalando.net/library/alpine-3:latest . | ||
| --build-arg BASE_IMAGE=container-registry.zalando.net/library/alpine-3:latest . |
There was a problem hiding this comment.
Similar to https://github.com/zalando/skipper/pull/2566/files#r1320057345. This is the right approach for targeting Zalando clusters but for the open source images (do you still publish to public registries?) alpine:latest would also work here.
There was a problem hiding this comment.
But like described above, it's perfectly fine to use a multi-arch base image (either alpine:latest or container-registry.zalando.net/library/alpine-3:latest) to build a single-arch image (i.e. --platform linux/amd64 or --platform linux/arm64).
There was a problem hiding this comment.
I'm assuming you're using docker buildx build. For the places where you're still using docker build (if any, I didn't check) the outcome is different. Better to stick with buildx everywhere.
| IMAGE=skipper-packaging-test:arm64 TARGETPLATFORM=linux/arm64 make test-run | ||
|
|
||
| ARM_IMAGE=skipper-packaging-test:armv7 make docker.build.armv7 | ||
| IMAGE=skipper-packaging-test:armv7 TARGETPLATFORM=linux/arm/v7 make test-run |
There was a problem hiding this comment.
Should/Can we also test the multi-arch image? Maybe even by running it on different (emulated) archs?
There was a problem hiding this comment.
I omitted multiarch here because it is not possible to build it without pushing docker/buildx#59. I guess to test it we'd need to push it first.
Makefile changes == Build local architecture binaries into build/ folder and nest other architecture-dependent binaries into build/<arch> subfolders (e.g. build/linux/amd64). Add help and make it a default target. Cleanup tar packages. Remove unused variables. Dockerfile changes == Remove BUILD_FOLDER build arg, see Makefile changes. Use COPY instead of ADD following https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#add-or-copy Remove redundant commands. Use one Docker file for all architectures. Use alpine:latest in Dockerfile and configure trusted base images for amd64 and multiarch builds in Makefile. Testing == Using https://github.com/multiarch/qemu-user-static it is possible to test that all make targets produce images and binaries for proper architectures, see `make test` Links == * https://www.docker.com/blog/multi-arch-images/ * https://www.thorsten-hans.com/how-to-build-multi-arch-docker-images-with-ease/ * https://www.stereolabs.com/docs/docker/building-arm-container-on-x86/ * https://github.com/multiarch/qemu-user-static * https://stackoverflow.com/questions/72444103/what-does-running-the-multiarch-qemu-user-static-does-before-building-a-containe Signed-off-by: Alexander Yastrebov <alexander.yastrebov@zalando.de>
AlexanderYastrebov
force-pushed
the
packaging/refactor-docker-build
branch
from
e7ea94b
to
785678b
Compare
|
👎 |
Makefile changes
Build local architecture binaries into build/ folder and nest other architecture-dependent binaries into build/ subfolders (e.g. build/linux/amd64).
Add help and make it a default target.
Cleanup tar packages.
Remove unused variables.
Dockerfile changes
Remove BUILD_FOLDER build arg, see Makefile changes.
Use COPY instead of ADD following https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#add-or-copy
Remove redundant commands.
Use one Docker file for all architectures.
Use alpine:latest in Dockerfile and configure trusted base images for amd64 and multiarch builds in Makefile.
Testing
Using https://github.com/multiarch/qemu-user-static it is possible to test that all make targets produce images and binaries for proper architectures, see
make testLinks