Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docker: all dependencies are reinstalled on each run #37

Open
VallariAg opened this issue Nov 7, 2023 · 6 comments · May be fixed by #39
Open

docker: all dependencies are reinstalled on each run #37

VallariAg opened this issue Nov 7, 2023 · 6 comments · May be fixed by #39
Labels
good first issue Good for newcomers

Comments

@VallariAg
Copy link
Member

current behaviour: all dependencies of teuthology-api (configured in setup.cfg) are reinstalled everytime docker container is started

expected behaviour: install dependencies before copying files (so changes in file won't require re-intalling dependencies) or cache
@VallariAg VallariAg added the good first issue Good for newcomers label Nov 7, 2023
@VallariAg VallariAg changed the title docker: all dependencies are installed again on each run docker: all dependencies are reinstalled on each run Nov 7, 2023
@dikwickley
Copy link

hey @VallariAg I would like to work on this.

@VallariAg
Copy link
Member Author

@dikwickley sure, go ahead!

@dikwickley
Copy link

dikwickley commented Nov 8, 2023 •
edited

I tried to reproduce this but can't seem to do it.

aniket@air teuthology-api % docker build -t t-api .

[+] Building 314.3s (11/11) FINISHED                                                                                           docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                           0.0s
 => => transferring dockerfile: 530B                                                                                                           0.0s
 => [internal] load .dockerignore                                                                                                              0.0s
 => => transferring context: 2B                                                                                                                0.0s
 => [internal] load metadata for docker.io/library/ubuntu:focal                                                                                3.8s
 => [1/6] FROM docker.io/library/ubuntu:focal@sha256:ed4a42283d9943135ed87d4ee34e542f7f5ad9ecf2f244870e23122f703f91c2                         10.6s
 => => resolve docker.io/library/ubuntu:focal@sha256:ed4a42283d9943135ed87d4ee34e542f7f5ad9ecf2f244870e23122f703f91c2                          0.0s
 => => sha256:ed4a42283d9943135ed87d4ee34e542f7f5ad9ecf2f244870e23122f703f91c2 1.13kB / 1.13kB                                                 0.0s
 => => sha256:a80d11b67ef30474bcccab048020ee25aee659c4caaca70794867deba5d392b6 424B / 424B                                                     0.0s
 => => sha256:0341906bdafc976cd73b05ea0e3df2e4884c6b6816197a2ffbd2367061c19acf 2.32kB / 2.32kB                                                 0.0s
 => => sha256:915eebb74587f0e5d3919cb77720c143be9a85a8d2d5cd44675d84c8c3a2b74a 25.97MB / 25.97MB                                               9.8s
 => => extracting sha256:915eebb74587f0e5d3919cb77720c143be9a85a8d2d5cd44675d84c8c3a2b74a                                                      0.6s
 => [internal] load build context                                                                                                              6.9s
 => => transferring context: 511.35MB                                                                                                          6.7s
 => [2/6] RUN apt-get update &&     apt-get install -y     git     qemu-utils     python3-dev     libssl-dev     ipmitool     python3-pip     83.7s
 => [3/6] COPY .teuthology.yaml /root                                                                                                          0.0s
 => [4/6] WORKDIR /teuthology_api                                                                                                              0.0s 
 => [5/6] COPY . /teuthology_api/                                                                                                              4.1s 
 => [6/6] RUN pip3 install -e .                                                                                                              202.6s 
 => exporting to image                                                                                                                         9.4s 
 => => exporting layers                                                                                                                        9.4s 
 => => writing image sha256:6ea4f5746ffc7e4b939f129f62e65c3d4ff9e8df485dfac00bd0482e4782a616                                                   0.0s 
 => => naming to docker.io/library/t-api                                                                                                       0.0s

What's Next?
  View a summary of image vulnerabilities and recommendations → docker scout quickview
aniket@air teuthology-api % docker images ls
REPOSITORY   TAG       IMAGE ID   CREATED   SIZE
aniket@air teuthology-api % docker image ls 
REPOSITORY    TAG       IMAGE ID       CREATED          SIZE
t-api         latest    6ea4f5746ffc   20 seconds ago   1.71GB
hello-world   latest    b038788ddb22   6 months ago     9.14kB
aniket@air teuthology-api % docker run -it 6ea
+ trap exit TERM
+ HOST=0.0.0.0
+ PORT=8080
+ cd /teuthology_api/src/
+ [  = development ]
+ gunicorn -c /teuthology_api/gunicorn_config.py teuthology_api.main:app
[2023-11-08 13:03:01 +0000] [8] [INFO] Starting gunicorn 21.2.0
[2023-11-08 13:03:01 +0000] [8] [INFO] Listening at: http://0.0.0.0:8080 (8)
[2023-11-08 13:03:01 +0000] [8] [INFO] Using worker: uvicorn.workers.UvicornWorker
[2023-11-08 13:03:02 +0000] [10] [INFO] Booting worker with pid: 10
[2023-11-08 13:03:02 +0000] [11] [INFO] Booting worker with pid: 11
[2023-11-08 13:03:02 +0000] [12] [INFO] Booting worker with pid: 12
[2023-11-08 13:03:02 +0000] [13] [INFO] Booting worker with pid: 13
[2023-11-08 13:03:02 +0000] [14] [INFO] Booting worker with pid: 14
[2023-11-08 13:03:02 +0000] [15] [INFO] Booting worker with pid: 15
[2023-11-08 13:03:02 +0000] [16] [INFO] Booting worker with pid: 16
[2023-11-08 13:03:02 +0000] [17] [INFO] Booting worker with pid: 17
[2023-11-08 13:03:04 +0000] [13] [INFO] Started server process [13]
[2023-11-08 13:03:04 +0000] [13] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [16] [INFO] Started server process [16]
[2023-11-08 13:03:04 +0000] [12] [INFO] Started server process [12]
[2023-11-08 13:03:04 +0000] [12] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [16] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [14] [INFO] Started server process [14]
[2023-11-08 13:03:04 +0000] [13] [INFO] Application startup complete.
[2023-11-08 13:03:04 +0000] [14] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [12] [INFO] Application startup complete.
[2023-11-08 13:03:04 +0000] [16] [INFO] Application startup complete.
[2023-11-08 13:03:04 +0000] [14] [INFO] Application startup complete.
[2023-11-08 13:03:04 +0000] [11] [INFO] Started server process [11]
[2023-11-08 13:03:04 +0000] [11] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [15] [INFO] Started server process [15]
[2023-11-08 13:03:04 +0000] [15] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [10] [INFO] Started server process [10]
[2023-11-08 13:03:04 +0000] [11] [INFO] Application startup complete.
[2023-11-08 13:03:04 +0000] [10] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [15] [INFO] Application startup complete.
[2023-11-08 13:03:04 +0000] [10] [INFO] Application startup complete.
[2023-11-08 13:03:04 +0000] [17] [INFO] Started server process [17]
[2023-11-08 13:03:04 +0000] [17] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [17] [INFO] Application startup complete.

As you can see, the dependencies were installed during the docker build step.
After the image is built, running the container in interactive mode does not reinstall the dependencies.

Were you referring to caching the dependencies during the build step? or am i missing something while trying to reproduce this.

@VallariAg
Copy link
Member Author

Yes, I am referring to caching them during the build step. Any changes in the root directory of teuthology-api will lead to reinstalling of all dependencies in the next build.
This might be helpful: https://docs.docker.com/build/cache/

@dikwickley
Copy link

Alright got it.

@dikwickley dikwickley linked a pull request Nov 9, 2023 that will close this issue
Open
@Devansh3712
Copy link
Member

I searched about caching pip dependencies so it only re-installs if there is a change in requirements.txt, this might be useful in our use case - https://stackoverflow.com/a/57282479

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first issue Good for newcomers
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Docker dependencies cache dikwickley/teuthology-api
3 participants
@dikwickley @VallariAg @Devansh3712