Compose v3: Moved shared volume to top-level volumes entry #709

sjawhar · 2017-02-09T00:40:36Z

For compatibility for Docker Compose v3, replaced the unnamed volume in the nginx service with a named nginx_conf volume that is shared by both services.

https://docs.docker.com/compose/compose-file/#/upgrading

For compatibility for Docker Compose v3, replaced the unnamed volume in the nginx service with a named nginx_conf volume that is shared by both services.

endersonmaia · 2017-02-24T18:44:06Z

I get this error :

myservice_dockergen.1.07txurxupiwb@moby    | 2017/02/23 22:13:47 Cnable to create temp file: open /etc/nginx/conf.d/docker-gen327150582: no such file or directory

sjawhar · 2017-02-25T03:55:16Z

@endersonmaia That's strange, running docker-compose -f docker-compose-separate-containers.yml up -d works flawlessly for me on both Windows and CentOS 7. Are you running docker 1.13.0+? That's required for compose v3.

teohhanhui · 2017-04-12T05:07:25Z

We have no real way of controlling which base image the files are copied from. So this may or may not work sometimes...

UPDATE: Actually, we can use the nocopy mount option.

teohhanhui · 2017-04-12T05:37:06Z

docker-compose-separate-containers.yml

    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
      - ./nginx.tmpl:/etc/docker-gen/templates/nginx.tmpl
+      - nginx_conf:/etc/nginx/conf.d


- nginx-conf:/etc/nginx/conf.d:nocopy

This is to prevent Docker from copying the contents from the wrong container.

Can you explain what you mean by "the wrong container"?

Docker copies files from the image into the volume when starting the container.

If files are copied from the dockergen container into the nginx-conf volume, it'd become empty. This would happen when the dockergen container is started before the nginx container. Adding nocopy here ensures that files are never copied from this container.

I just tested it, and the volume is not emptied if it has data. Considering that the /etc/nginx/conf.d folder does not exist in jwilder/docker-gen, and considering that docker-gen is maintaining the contents of this volume anyway, can you explain your concern?

Build an nginx image with your own configuration files in /etc/nginx/conf.d.

Delete the containers and the volume.

Start the dockergen service.

Start the nginx service.

You'll see that the nginx-conf volume was initialized to empty from the dockergen image. Then it'll only have one default.conf after generation from template. Your own configuration files would be lost.

Did you make sure to start the dockergen service before the nginx service? Because startup order is not guaranteed, and that's why the problem could happen sometimes.

I was finally able to reproduce this issue, but adding nocopy doesn't fix it. If the problem is startup order, maybe the right solution is to add depends_on to the dockergen service?

Are you sure adding nocopy doesn't fix it? Because it works for me. You'll need to make sure to remove the volumes before each run to be able to test it properly.

depends_on has no effect when using docker stack deploy, so it's not a solution. (Basically we should not care about the startup order.) See moby/moby#31333

If I just run docker-compose up, the config files are there regardless of whether nocopy is added to the dockergen service. This is probably just luck, given the race condition you describe, though I tried a few dozen times.

If I run docker-compose up dockergen then docker-compose up nginx, the config files are missing regardless of whether nocopy is added to the dockergen service.

Is there some other test I can run to verify the behavior you're describing?

I'm starting to think this is not actually an issue. If I need to make a custom Nginx image, couldn't I just place the config file somewhere other than /etc/nginx/conf.d? Problem solved.

Alternatively, we could use a /etc/nginx/dockergen folder, but then you'd need a derived Nginx image.

teohhanhui · 2017-04-12T05:41:39Z

docker-compose-separate-containers.yml


  whoami:
    image: jwilder/whoami
    environment:
      - VIRTUAL_HOST=whoami.local
+
+volumes:
+  nginx_conf: {}


nginx-conf, because that's the convention used by Docker.

For example here: https://docs.docker.com/engine/tutorials/dockervolumes/#mount-a-shared-storage-volume-as-a-data-volume (my-named-volume)

Considering that Docker Compose names everything with an underscore ($PROJECT_default network, $PROJECT_$SERVICE_1 container, etc.), I'm not so sure about "convention." Doing that results in a volume called <project_name>_nginx-conf, which is just... ugly.

That's intended. The underscore is used for prefix (e.g. project name) and postfix (e.g. _default, _1). Using dash means it doesn't get mixed together with the prefix and postfix parts.

I don't agree. This seems like an issue of standards. Maybe the project maintainers have an opinion? @jwilder

teohhanhui · 2017-04-12T05:42:56Z

docker-compose-separate-containers.yml

@@ -1,23 +1,25 @@
-version: '2'
+version: '3'


No need to bump the version here, so that it can still work with older Docker and Docker Compose versions.

You're right, no reason to rush to v3 until other Stack compatibility issues are fixed.

Though version 3 is list as the recommended version
https://docs.docker.com/compose/compose-file/compose-versioning/

Compose v3: Moved shared volume to top-level volumes entry

b680816

For compatibility for Docker Compose v3, replaced the unnamed volume in the nginx service with a named nginx_conf volume that is shared by both services.

sjawhar mentioned this pull request Apr 8, 2017

fix template when running docker-gen and nginx in separate containers #750

Open

teohhanhui suggested changes Apr 12, 2017

View reviewed changes

sjawhar added 2 commits April 12, 2017 05:39

Set compose file version back to 2.1 for separate containers

d85c033

Merge branch 'master' into patch-1

8ca9cb0

sjawhar closed this Feb 20, 2018

sjawhar deleted the patch-1 branch February 20, 2018 17:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compose v3: Moved shared volume to top-level volumes entry #709

Compose v3: Moved shared volume to top-level volumes entry #709

sjawhar commented Feb 9, 2017

endersonmaia commented Feb 24, 2017

sjawhar commented Feb 25, 2017 •

edited

teohhanhui commented Apr 12, 2017 •

edited

teohhanhui Apr 12, 2017 •

edited

sjawhar Apr 12, 2017

teohhanhui Apr 13, 2017

sjawhar Apr 13, 2017 •

edited

teohhanhui Apr 13, 2017

teohhanhui Apr 17, 2017

sjawhar Apr 17, 2017

teohhanhui Apr 17, 2017 •

edited

sjawhar Apr 17, 2017 •

edited

sjawhar Apr 17, 2017 •

edited

teohhanhui Apr 12, 2017

sjawhar Apr 12, 2017 •

edited

teohhanhui Apr 13, 2017 •

edited

sjawhar Apr 13, 2017

teohhanhui Apr 12, 2017

sjawhar Apr 12, 2017

sjawhar Apr 12, 2017 •

edited

Compose v3: Moved shared volume to top-level volumes entry #709

Compose v3: Moved shared volume to top-level volumes entry #709

Conversation

sjawhar commented Feb 9, 2017

endersonmaia commented Feb 24, 2017

sjawhar commented Feb 25, 2017 • edited

teohhanhui commented Apr 12, 2017 • edited

teohhanhui Apr 12, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sjawhar Apr 13, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

teohhanhui Apr 17, 2017 • edited

Choose a reason for hiding this comment

sjawhar Apr 17, 2017 • edited

Choose a reason for hiding this comment

sjawhar Apr 17, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sjawhar Apr 12, 2017 • edited

Choose a reason for hiding this comment

teohhanhui Apr 13, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sjawhar Apr 12, 2017 • edited

Choose a reason for hiding this comment

sjawhar commented Feb 25, 2017 •

edited

teohhanhui commented Apr 12, 2017 •

edited

teohhanhui Apr 12, 2017 •

edited

sjawhar Apr 13, 2017 •

edited

teohhanhui Apr 17, 2017 •

edited

sjawhar Apr 17, 2017 •

edited

sjawhar Apr 17, 2017 •

edited

sjawhar Apr 12, 2017 •

edited

teohhanhui Apr 13, 2017 •

edited

sjawhar Apr 12, 2017 •

edited