New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use docker compose for building and running container (local and CI) (github action and circleCI) #22211
Conversation
844030d
to
a1aad55
Compare
539d877
to
eaeecfb
Compare
04c61ed
to
a05de89
Compare
cdf27df
to
f2fc6a7
Compare
b0be9bf
to
4d5a3d2
Compare
I've found some issues with the make file variables that I'd like to address before merging this. I will open a separate small PR with some testing to ensure changes here and future changes are valid and sound. |
3405724
to
86b61b5
Compare
|
4e0b8f0
to
6a8f7e8
Compare
- base for global definition - build to define build config - mount deps to expose /deps in container on ./deps - mount olympia to mount repository root - simplify extract deps config
fe1fd91
to
610fa58
Compare
Relates to: https://mozilla-hub.atlassian.net/browse/ADDSRV-720
Relates to: https://mozilla-hub.atlassian.net/browse/AMO2-599
Partially fulfills: #22182
Description
This PR aligns the way we build and run our docker image/container in local and CI environments. This reduces the amount of configuration needed and streamlines execution of CI.
This is primarily achieved through using
buildx bake
to build the docker image, allowing much of our build configuration to be expressed in a docker-compose.yml file.Secondly, we are now executing our run commands in CI using docker compose, simplifying the run config (same reasons as above) and ensuring we run locally and in CI the same way.
Context
This PR enables follow up PRs to expand on docker compose in CI allowing us to run dependent services and expand coverage with tests etc.
To allow environment specific execution of a particular version, the docker version is now expressed via an environment variable, instead of hard coded to
latest
.DOCKER_VERSION
Local dev
The default values for these are correct for running a local build of the image
mozilla/addons-server:local
This build is now heavily cached and arguably faster and more bandwidth efficient over time than downloading the image periodically.
It is also infinitely more reliable in determine if your local code will actually run in CI or production because you develop with a locally built image.
CI
Now in CI we follow the same workflow for both building and running the image. we can specify the DOCKER_VERSION to build and run the image in CI allowing tight control over the specific version we are testing (pr-<PR_NUMBER>)
Using docker compose to run the image is both necessary for future migration of our CI to github action but also convenient now so we can infer the image tag via .env already.
Testing
You can test these changes locally and in CI.
Local
Build the image
Build the image with a custom tag
Testing
docker-compose.ci.yml
run
echo "COMPOSE_FILE=docker-compose.ci.yml" >> .env
to switch to "CI" mode.Now if you
docker_compose_up
you should no longer have a mount for ./deps or for the . root directory.Try creating a file in the repo root. Now check if that file exists in the container.. should not.
You can try the same thing for
docker-compose.olympia.yml
this will add a mount for the repo root but not for ./deps.You can change the docker target
DOCKER_TARGET
in the .env file and re up the containers, if you set the target toproduction
and have a clean ./deps directory or are using the ci.yml file, the container should fail due to missing supervisor which is a dev dependency.CI
Current workflows are using the new approach so green means good.