Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Execute a command after run #1809

Closed
ahmet2mir opened this issue Aug 5, 2015 · 147 comments
Closed

Execute a command after run #1809

ahmet2mir opened this issue Aug 5, 2015 · 147 comments

Comments

@ahmet2mir
Copy link

ahmet2mir commented Aug 5, 2015

Hi,

It will be very helpful to have something like "onrun" in the YAML to be able to run commands after the run. Similar to moby/moby#8860

mongodb:
    image: mongo:3.0.2
    hostname: myhostname
    domainname: domain.lan
    volumes:
        - /data/mongodb:/data
    ports:
        - "27017:27017" 
    onrun:
        - mongodump --host db2dump.domain.lan --port 27017 --out /data/mongodb/dumps/latest
        - mongorestore -d database /data/mongodb/dumps/latest/database

After the mongodb start, It will dump db2dump.domain.lan and restore it.

When I will stop and then start the container, onrun part will no be executed to preserve idempotency.

EDIT 15 June 2020

5 years later, Compose wan't to "standardize" specifications,
please check compose-spec/compose-spec#84

@dnephin
Copy link

dnephin commented Aug 5, 2015

I think these should be steps in the Dockerfile

FROM mongo:3.0.2
ADD data/mongodb/dumps/latest /data/mongodb/dumps/latest
RUN mongorestore -d database /data/mongodb/dumps/latest/database

That way you also get it cached when you rebuild.

@ahmet2mir
Copy link
Author

Thanks @dnephin.
Of course I can make a Dockerfile and use it in build instead of images, or I can use docker exec.
MongoDB is just an example, you can have this example with mysql and account creation, or with rabbitmq and queue creation etc.

onrun will permits flexibility on compose orchestration, compose will read onrun list and make docker exec on each item.

@aanand
Copy link

aanand commented Aug 5, 2015

The point is that putting commands to docker exec in docker-compose.yml is unnecessary when you can either do it in the Dockerfile or in the container's startup script, both of which will also make your container more useful when not being run with Compose.

Alternatively, start your app with a shell script or Makefile that runs the appropriate docker and docker-compose commands.

The functionality isn't worth adding to Compose unless it would add significant value over doing either of those, and I don't think it would for the use cases you've cited.

@ahmet2mir
Copy link
Author

So, to manage my docker, you suggest me to use a Script or a Makefile. So why compose was created
? We can manage, scale etc. container with script || dockerfile ?

Ok, I take this example, it's what I used to deploy my application testing environment in the CI process.

rabbitmq:
    image: rabbitmq:3.5.1-management
    environment:
        RABBITMQ_NODENAME: rabbit
    hostname: rabbitmq
    domainname: domain.lan
    volumes:
        - /data/rabbitmq/db:/var/lib/rabbitmq
    ports:
        - "5672:5672" 
        - "15672:15672"
        - "25672:25672"
        - "4369:4369"

mongodb:
    image: mongo:3.0.2
    hostname: mongo
    domainname: domain.lan
    volumes:
        - /data/mongodb:/data
    ports:
        - "27017:27017" 

appmaster:
    image: appmaster
    hostname: master
    domainname: domain.lan
    environment:
        ...
    ports:
        - "80:80" 
        - "8080:8080"
    links:
        - mongodb
        - rabbitmq

celery:
    image: celery
    hostname: celery
    domainname: domain.lan
    environment:
        ...
    links:
        - rabbitmq

After container starts, I must provision mongodb, manage queue and account in rabbitmq

What i'm doing today is a script with:

#!/bin/bash
PROJECT=appmaster
docker-compose -f appmaster.yml -p appmaster up -d
docker exec appmaster_rabbitmq_1 rabbitmqctl add_user user password
docker exec appmaster_rabbitmq_1 rabbitmqctl add_vhost rabbitmq.domain.lan
docker exec appmaster_rabbitmq_1 rabbitmqctl set_permissions -p rabbitmq.domain.lan password ".*" ".*" ".*"
docker exec appmaster_mongodb_1 mongodump --host mongo-prd.domain.lan --port 27017 --out /data/mongodb/dumps/latest
docker exec appmaster_mongodb_1 mongorestore -d database /data/mongodb/dumps/latest/database

With onrun instruction I can directly make docker-compose -f appmaster.yml -p appmaster up -d
and the yml file become more readable

rabbitmq:
    ...
    onrun:
        - rabbitmqctl add_user user password
        - rabbitmqctl add_vhost rabbitmq.domain.lan
        - rabbitmqctl set_permissions -p rabbitmq.domain.lan password ".*" ".*" ".*"

mongodb:
    ...
    onrun:
        - mongodump --host mongo-prd.domain.lan --port 27017 --out /data/mongodb/dumps/latest
        - mongorestore -d database /data/mongodb/dumps/latest/database

@sthulb
Copy link

sthulb commented Aug 21, 2015

This would be rather useful and solves a use case.

@sunsided
Copy link

👍

@hypergig
Copy link

It will make using docker-compose more viable for gated tests as part of a CD pipeline

@fferraris
Copy link

👍

@dnephin
Copy link

dnephin commented Oct 16, 2015

This is a duplicate of #877, #1341, #468 (and a few others).

I think the right way to support this is #1510 and allow external tools to perform operations when you hit the event you want.

Closing as a duplicate

@dnephin dnephin closed this as completed Oct 16, 2015
@jasonrhaas
Copy link

This would be very useful. I don't understand the argument of "oh you could do this with a bash script". Of course we could do it with a bash script. I could also do everything that Docker-compose does with a bash script. But the point is that there is one single YAML file that controls your test environment and it can be spun up with a simple docker-compose up command.

@aanand
Copy link

aanand commented Aug 1, 2016

It is not the remit of Compose to do everything that could be done with a shell script or Makefile - we have to draw a line somewhere to strike a balance between usefulness and avoiding bloat.

Furthermore, one important property of the Compose file is that it's pretty portable across machines - even Mac, Linux and Windows machines. If we enable people to put arbitrary shell commands in the Compose file, they're going to get a lot less portable.

@sunsided
Copy link

sunsided commented Aug 1, 2016

@aanand To be fair, being able to execute a docker exec does not automatically imply x-plat incompatibility.

@aanand
Copy link

aanand commented Aug 1, 2016

Apologies - I misread this issue as being about executing commands on the host machine. Still, my first point stands.

@jasonrhaas
Copy link

I understand your point @aanand. It doesn't seem out of scope to me, since already docker-compose does a lot of the same things that the regular docker engine already does, like command, expose, ports, build, etc. Adding the exec functionality would add more power to docker-compose to make it a true one stop shop for setting up dev environments.

@ahmet2mir
Copy link
Author

ahmet2mir commented Aug 2, 2016

@aanand the main problem for many devs and CI pipelines is to have a data very close to the production env. Like a dump from a DB. I create this ticket 1 year ago and nothing move in docker compose.

So you suggest a Makefile or a Bashcript just to run some exec #1809 (comment)

What I originally suggest is onrun (or oncreate) who keep idempotency. Just run at the first start. If the container is stopped or paused, the new start will not run onrun (or oncreate)

Finally, in my git repository I will have a compose file, a dockerfile and a makefile with idempotency management (may makefile could create a statefile). Genius!

@dnephin
Copy link

dnephin commented Aug 2, 2016

There's a big difference between command, expose, etc and exec. The first group are container options, exec is a command/api endpoint. It's a separate function, not options to the create container function.

There are already a couple ways to accomplish this with Compose (#1809 (comment)). onrun already exists. It is command.

Regarding the specific problem of dumping or loading data from a database, those are more "workflow" or "build automation" type tasks, that are generally done in a Makefile. I've been prototyping a tool for exactly those use-cases called dobi, which runs all tasks in containers. It also integrates very well with Compose. You might be interested in trying it out if you aren't happy with Makefiles. I'm working on an example of a database init/load use case.

@ahmet2mir
Copy link
Author

ahmet2mir commented Aug 2, 2016

@dnephin onrun is not a simple command because you just miss the idempotency.

Let's imagine. create on container creation and will never be exec again (dump & restore).

exec:
    create:
        - echo baby
    destroy:
        - echo keny
    start:
        - echo start
    stop:
        - echo bye

If you need more examples:

Thanks for dobi, but if you need to create a tool to enhance compose, compose is bad and it's better to use a more powerfull tool.

@dnephin
Copy link

dnephin commented Aug 2, 2016

but if you need to create a tool to enhance compose, compose is bad and it's better to use a more powerful tool.

That's like saying "if you need applications to enhance your operating system, your OS is bad". No one tool should do everything. The unix philosophy is do one thing, and do it well. That is what we're doing here. Compose does its one thing "orchestrate containers for running an application". It is not a build automation tool.

@ahmet2mir
Copy link
Author

ahmet2mir commented Aug 3, 2016

That's like saying "if you need applications to enhance your operating system, your OS is bad". No one tool should do everything. The unix philosophy is do one thing, and do it well. That is what we're doing here.

Wow I think that we reached the best bad faith.

Unfortunately, a simple re-usable component is not how things are playing out. Docker now is building tools for launching cloud servers, systems for clustering, and a wide range of functions: building images, running images, uploading, downloading, and eventually even overlay networking, all compiled into one monolithic binary running primarily as root on your server. The standard container manifesto was removed. We should stop talking about Docker containers, and start talking about the Docker Platform. It is not becoming the simple composable building block we had envisioned.

So you can guarantee that we will never see "docker compose" wrote in Go inside in the docker monolithic binary to keep the unix philosophy ? https://www.orchardup.com/blog/orchard-is-joining-docker

To continue towards that original goal, we’re joining Docker. Among other things, we’re going to keep working on making Docker the best development experience you’ve ever seen – both with Fig, and by incorporating the best parts of Fig into Docker itself.

@discordianfish
Copy link

So in short there is no way to do things like loading fixtures with compose..? I have to say I'm surprised..
The official way is to add fixture loading to my production container? Or to write a shell script around my compose file? In the later case I could also just execute 'docker run' as I did before.

@dopry
Copy link

dopry commented Apr 7, 2017

@discordianfish, If, somehow, someone would wake up to the fact that CI/CD engineers need to be able to handle life cycle events and orchestration at least at a very basic level, then who knows docker/docker-compose may actually make its way out of local development pipelines and testing infrastructure and find a place in more production environments. I'm hopeful whoever is working on the stacks will address these issues, but I won't hold my breath.

After all what needs to be done at build time may be different than what is needed at runtime, and is needed at runtime often varies by deployment environment...

It is kind of annoying work to make my external scripts aware of whether an up is going to create or start containers...

And those are things some lifecycle hooks + commands + environment variables could help with.

You see it in service management frameworks and other orchestration tools... why not in docker-compose?

@dnephin
Copy link

dnephin commented Apr 8, 2017

You might be interested in https://github.com/dnephin/dobi , which is a tool I've been working on that was designed for those workflows.

@ahmet2mir
Copy link
Author

@dnephin stop spamming this issue with your tools. We see your comment before and the answer is the same. Makefile/bash is probably better than an nth "my tool enhance docker".

@dnephin
Copy link

dnephin commented Apr 10, 2017

Thank you for your constructive comment. I didn't realize that I had already mentioned dobi on this thread 8 months ago.

If you're happy with Makefile/bash that's great! I'm glad your problem has been solved.

@lucile-sticky
Copy link

Added a comment related to this topic here: #1341 (comment)

@lucile-sticky
Copy link

@dnephin for this one, my comment can be applied:

So sad that this issue have been closed because of some refractoriness to evolution 😞

The greatest value of having docker compose is standardization

That's the point. If we could "just" write a .sh file or whatever to do the job without using Docker Compose, why Docker Compose is existing? 😕

We can understand that is a big job, as @shin- said:

it's unfortunately too much of a burden to support at that stage of the project

❤️

But you can't say "Make a script" what means "Hey, that's too hard, we're not gonna make it".

If it's hard to do it, just say "Your idea is interesting, and it fills some needs, but it's really difficult to do and we don't have resources to do it at this time... Maybe could you develop it and ask a pull request" or something like that 💡

In #1341, I "only" see a way to write in docker-compose.yml commands like nmp install that would be run before or after some events (like container creation), like you would do with docker exec <container id> npm install for example.

Use case

I have a custom NodeJS image and I want to run npm install in the container created from it, with a docker-compose up --build.

My problem is: the application code is no added in the container, it's mounted in it with a volume, defined in docker-compose.yml:

custom-node:
    build: ../my_app-node/
    tty: true
    #command: bash -c "npm install && node"
    volumes:
     - /var/www/my_app:/usr/share/nginx/html/my_app

so I can't run npm install in the Dockerfile because it needs the application code to check dependencies. I described the behavior here: http://stackoverflow.com/questions/43498098/what-is-the-order-of-events-in-docker-compose

To run npm install, I have to use a workaround, the command statement:

command: bash -c "npm install && node"

which is not really clean 😞 and which I can't run on Alpine versions (they don't have Bash installed in it).

I thought that Docker Compose would provide a way to run exec commands on containers, e.G.:

custom-node:
    build: ../my_app-node/
    tty: true
    command: node
    volumes:
     - /var/www/my_app:/usr/share/nginx/html/my_app
    exec:
     - npm install

But it's not, and I think it's really missing!

@discordianfish
Copy link

I expected compose is designed for testing, but I'm probably wrong and it's intended more for local development etc. I ran into several other rough edges like orphaned containers and the unclear relation between project name, path and how it's used to identify ownership, what happens if you have multiple compose files in the same directory etc etc. So all in all it doesn't seem like a good fit for CI.
Instead I'm planning to reuse my production k8s manifests in CI by running kubelet standalone. This will also require lots of glue, but at least this way I can use the same declarations for dev, test and prod.

@dnephin
Copy link

dnephin commented Apr 20, 2017

@lucile-sticky you can use sh -c in alpine.

It sounds like what you want is "build automation" which is not the role of docker-compose. Have you looked at dobi ?

@lucile-sticky
Copy link

Two questions:

  • Why is this not the role of Docker Compose?
  • If the point is to have only one tool to rule them all, why would I use an other tool to complete a task that Docker Compose is not able to do?

@Strandedpirate
Copy link

k8s != docker-compose. Wrong channel

@jamespfennell
Copy link

Sorry for not being clear, but my point was: Kubernetes supports this, and because Kubernetes and Docker compose have many of the same use cases/purposes, that would be an argument for having it in compose. Sorry if I was unclear.

@alex-ubitec
Copy link

alex-ubitec commented Apr 9, 2020

Good news!!

I think docker has heard us, (on this issue and a few others). https://www.docker.com/blog/announcing-the-compose-specification/

Let's try to work on the specification there to fulfill the community needs. We can try to make this an open and friendly community with this restart.

@taythebot
Copy link

taythebot commented Jun 11, 2020

Good news!!

I think docker has heard us, (on this issue and a few others). https://www.docker.com/blog/announcing-the-compose-specification/

Let's try to work on the specification there to fulfill the community needs. We can try to make this an open and friendly community with this restart.

Has anyone suggested this change yet? Mailing list isn't available yet so I think the next best place is here: https://github.com/compose-spec/compose-spec

I don't see an issue that describes this problem but not sure if that's the right place...

Edit: I opened an issue at compose-spec/compose-spec#84. Please upvote it to show your support for the feature!

@reduardo7
Copy link

reduardo7 commented Jul 13, 2020

You can use the HEALTHCHECK to do something else like following example:

Code

Dockerfile

FROM ubuntu

COPY healthcheck.sh /healthcheck.sh
RUN chmod a+x /healthcheck.sh

HEALTHCHECK --interval=5s CMD /healthcheck.sh

CMD bash -c 'set -x; set +e; while true; do cat /test.txt; sleep 3; done'

healthcheck.sh

#/usr/bin/env bash

set -e

FIRST_READY_STATUS_FLAG='/tmp/.FIRST_READY_STATUS_FLAG'

# Health check

echo 'Run command to validate the container status HERE'

# On success
if [ ! -f "${FIRST_READY_STATUS_FLAG}" ]; then
  # On first success...
  touch "${FIRST_READY_STATUS_FLAG}"

  # Run ON_RUN on first health check ok
  if [ ! -z "${DOCKER_ON_RUN}" ]; then
    eval "${DOCKER_ON_RUN}"
  fi
fi
  1. Run the health check.
    • If it fails, exits from script with exit code 1.
    • If the health check is ok, the script will continue.
  2. If it is the first health check OK and if DOCKER_ON_RUN environment variable exists, execute it.

Example

docker-compose.yml

version: "3.7"

services:
  test:
    build:
      context: .
    image: test/on-run
    environment:
      DOCKER_ON_RUN: echo x >> /test.txt

You can use DOCKER_ON_RUN environment variable to pass a custom command to execute after run.

Execution result

docker-compose build
docker-compose up

Output:

Creating network "tmp_default" with the default driver
Creating tmp_test_1 ... done
Attaching to tmp_test_1
test_1  | + set +e
test_1  | + true
test_1  | + cat /test.txt
test_1  | cat: /test.txt: No such file or directory
test_1  | + sleep 3
test_1  | + true
test_1  | + cat /test.txt
test_1  | cat: /test.txt: No such file or directory
test_1  | + sleep 3
test_1  | + true
test_1  | + cat /test.txt
test_1  | x
test_1  | + sleep 3
test_1  | + true
test_1  | + cat /test.txt
test_1  | x
test_1  | + sleep 3
test_1  | + true
test_1  | + cat /test.txt
test_1  | x
test_1  | + sleep 3
  • You can see the error cat: /test.txt: No such file or directory until the health check is ready.
  • You can see only one x inside /test.txt after run.

Hope this can help someone.

Edit 1

If you don't need a health check, you can use the rest of the script.

Edit 2

See https://github.com/reduardo7/docker-on-ready for examples

@augustgerro
Copy link

@reduardo7
Thanks for your workaround.
Just want to add, in case if your needs to run command one, like for users creation or etc, your can mount volume for the touch "${FIRST_READY_STATUS_FLAG}"

@web-ted
Copy link

web-ted commented Jul 16, 2020

Many of these solutions are valid workarounds to this problem. For e.g. making an entrypoint script could also resolve this:
ENTRYPOINT ["./entrypoint.sh"]

which will include a more complex logic before running the actual service or process.
This is still not a hook though that would allow us to inject logic in the container lifecycle:

  • before creating
  • before starting
  • after starting
  • before destroying
  • even after destroying
  • etc ...

I know that not all the above are meaningful but I hope that you get the picture because this is the point.
This could also be included in docker-compose with a directive like:

lifecycle:
    before_start: "./beforeStartHook.sh"
    after_destroy: "./afterDestroyHook.sh"

or even like that:

hooks:
    before_destroy: "./beforeDestroyHook.sh"
    before_create: "./fixFsRights.sh"

@devendraap
Copy link

I am unable to overwrite file which requires root permission using hook script or bootstrap script approach, since we start container as non root user

@zerthimon
Copy link

Wow, such a basic functionality and still not implemented.

@grandmaestr
Copy link

grandmaestr commented Dec 5, 2020

This is workaround from @reduardo7 worked for me.

You can use the HEALTHCHECK to do something else like following example:

Code

Dockerfile

FROM ubuntu

COPY healthcheck.sh /healthcheck.sh
RUN chmod a+x /healthcheck.sh

HEALTHCHECK --interval=5s CMD /healthcheck.sh

CMD bash -c 'set -x; set +e; while true; do cat /test.txt; sleep 3; done'

I have the following Dockerfile:

FROM debian:buster

RUN apt-get update && apt-get --no-install-recommends -y install \
    ca-certificates \
    cpanminus \
    default-libmysqlclient-dev \
    fonts-dejavu \
    gcc \
    gettext \
    git \
    libc6-dev \
    libexpat1-dev \
    libfribidi-dev \
    libgd-dev \
    libxslt1-dev \
    libyaz-dev \
    make \
    perl \
    pkg-config \
    && rm -rf /var/lib/apt/lists/*

RUN adduser --disabled-password --gecos '' koha

USER koha

WORKDIR /home/koha

ADD --chown=koha:koha https://raw.githubusercontent.com/Koha-Community/Koha/v20.11.00/cpanfile .

ENV PERL_CPANM_OPT --local-lib-contained /home/koha/.local
RUN export PERL_CPANM_OPT="--quiet --metacpan --notest $PERL_CPANM_OPT" \
    && cpanm --installdeps . \
    && cpanm Readonly Array::Utils JSON::Validator@4.05 IO::Scalar \
    && cpanm Starman \
    && rm -rf /home/koha/.cpanm

RUN git clone --progress --depth 1 --branch v20.11.00 https://github.com/Koha-Community/Koha.git koha

COPY --chown=koha:koha koha-conf.xml.in etc/
COPY --chown=koha:koha log4perl.conf etc/

ENV KOHA_CONF /home/koha/etc/koha-conf.xml
ENV PERL5LIB /home/koha/koha:/home/koha/.local/lib/perl5
ENV PATH /home/koha/.local/bin:$PATH
ENV LANG C.UTF-8

EXPOSE 5000 5001

COPY docker-entrypoint.sh /usr/local/bin/

WORKDIR /home/koha/koha
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["starman", "--listen", ":5000", "--listen", ":5001"]

I need to run the following command (to reindex ElasticSearch) after the container starts:

docker exec koha perl koha/misc/search_tools/rebuild_elasticsearch.pl -d

but couldn't get it working with ENTRYPOINT or CMD. So wrote a bash script (script.sh)

#!/bin/bash
perl koha/misc/search_tools/rebuild_elasticsearch.pl

, made it executable and added this block to the Dockerfile

COPY script.sh /script.sh HEALTHCHECK --interval=5s CMD /script.sh

which works. Question is, since HEALTHCHECK will will run the script every 5 s, is there a way to make HEALTHCHECK run only once and quit?

@reduardo7
Copy link

reduardo7 commented Dec 5, 2020

@grandmaestr :

Question is, since HEALTHCHECK will will run the script every 5 s, is there a way to make HEALTHCHECK run only once and quit?

The HEALTHCHECK runs every 5s, it's a Docker feature. But, with FIRST_READY_STATUS_FLAG='/tmp/.FIRST_READY_STATUS_FLAG' (at healthcheck.sh) we prepare the HEALTHCHECK to run only once the first time.

You need to define perl koha/misc/search_tools/rebuild_elasticsearch.pl -d at DOCKER_ON_RUN env variable (at Dockerfile: ENV DOCKER_ON_RUN="perl koha/misc/search_tools/rebuild_elasticsearch.pl -d"), or your healthcheck.sh should be something like following:

#/usr/bin/env bash

set -e

FIRST_READY_STATUS_FLAG='/tmp/.FIRST_READY_STATUS_FLAG'

# Health check

echo 'Run command to validate the container status HERE'

# On success
if [ ! -f "${FIRST_READY_STATUS_FLAG}" ]; then
  # On first success...
  touch "${FIRST_READY_STATUS_FLAG}"

  # Run ON_RUN on first health check ok
  perl koha/misc/search_tools/rebuild_elasticsearch.pl -d
fi

@grandmaestr
Copy link

Thank you @reduardo7, I'll try this out tomorrow and let you know how it goes. Much appreciated!

@reduardo7
Copy link

I created a repository with examples about how to implement this required feature:

https://github.com/reduardo7/docker-on-ready

@Luc45
Copy link

Luc45 commented Dec 9, 2020

Why this has to be so complicated?

@fescobar
Copy link

fescobar commented Dec 9, 2020

@Luc45 this is the next challenge after landing in Mars

@alexrecuenco
Copy link

@Luc45 this is the next challenge after landing in Mars

It will certainly take a similar amount of time ;)

@superswan
Copy link

This would be a nice feature. I have a case where I need to run a single command after starting PHP service and don't want to add a bash script wrapper or anything like that.

@imod
Copy link

imod commented Sep 24, 2021

@superswan not sure (and also never tried), but is this working for you: #1510 (comment)

@henryclw
Copy link

henryclw commented May 4, 2022

It's 2022, and yet we don't have an elegant way to do this.

@Ordiel
Copy link

Ordiel commented Jun 7, 2022

@henryclw have you check on command

@henryclw
Copy link

henryclw commented Jun 8, 2022

@henryclw have you check on command

This is not we want, as #1809 (comment) said, we want something that could only be executed once, just like initializing the database or run apt-get update && apt-get upgrade

@Strandedpirate
Copy link

image

@vysogot
Copy link

vysogot commented Mar 8, 2023

March 2023 and I just wanted to alter a user in a docker-compose after I up some database... but fine, bash script!
Nice chat guys! Although, one thing that makes using a script after running docker-compose yuck is that now I have to refer to a container by name. An arbitrary name from the script point of view (unless I go for some global env vars which I won't).

@ndeloof
Copy link
Contributor

ndeloof commented Mar 8, 2023

Those interested with this topic could help us providing feedback on compose-spec/compose-spec#289

@dopry
Copy link

dopry commented Mar 9, 2023

The solution I've landed at for at least the docker compose up scenarios are task containers that setup my databases etc. Here is an example froma django environment

# re-usable yaml anchors
x-task: &task
  # docker stack deploy restart policy for tasks
  deploy:
    restart_policy:
      condition: on-failure
    replicas: 1
  # docker compose stand-alone restart policy for tasks
  restart: on-failure

x-task-ensure-database: &task-ensure-database
  <<: *task
  # image:  this task must extend a service with a postgres image.
  entrypoint: |
    sh -c "
    psql -v ON_ERROR_STOP=0 --host postgres --username \"$${POSTGRES_USER}\" <<EOSQL
      CREATE ROLE $${SERVICE_NAME} LOGIN PASSWORD '$${SERVICE_PASS}';
      CREATE DATABASE $${SERVICE_NAME};
      GRANT ALL ON DATABASE $${SERVICE_NAME} TO $${SERVICE_NAME};
    EOSQL"
  # you must set the following environment variables where you use this anchor.
  # environment:
    # - POSTGRES_USER
    # - PGPASSWORD - superuser password
    # USERNAME and DATABASE to create.
    # - SERVICE_NAME=UserAndDbName
    # - SERVICE_PASS=password

x-service: &service
  # docker stack deploy restart policy for tasks
  deploy:
    restart_policy:
      condition: unless-stopped
    replicas: 1
  # docker compose stand-alone restart policy for tasks
  restart: unless-stopped

x-postgres: &postgres
    image: postgres:14
    environment:
      - POSTGRES_USER=example
      - POSTGRES_PASSWORD=example

volumes:
  postgres-data:
  cms-media:

services:
 postgres:
    <<: *service
    <<: *postgres
    # we're only exposing here, since this is only called from another service.
    expose:
      - "5432"
    # uncomment below if you wish to access this db directly from the host.
    ports:
      - "5432:5432"
    volumes:
      - postgres-data:/var/lib/postgresql

  api-ensure-database:
    <<: *postgres
    <<: *task-ensure-database
    environment:
      - POSTGRES_USER=example
      - PGPASSWORD=example
      - SERVICE_NAME=api
      - SERVICE_PASS=password
    depends_on:
     - postgres

api-ensure-database will keep trying to run until it succeeds. The 'task' services fail and respawn a lot while waiting on their dependencies which isn't ideal, but it does get the job done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests