Cannot update nginx config: device or resource busy

[hustlahusky]

Hi

I use jwilder/docker-gen docker image alongside with nginx docker image for proxying containers with other web services on my local development environment (Docker for Windows). This combination worked great for a long time. But after recent Docker for Windows update (current stable version 2.2.0.0) I start getting errors in jwilder/docker-gen container:

Unable to create dest file /etc/nginx/conf.d/default.conf: rename /etc/nginx/conf.d/docker-gen728953019 /etc/nginx/conf.d/default.conf: device or resource busy

I tried to boot into container tty instead of running docker-gen script and move file manually and it works without errors.

My docker-compose config:

version: '3'

services:
  nginx:
    image: nginx:1.17
    container_name: ld-proxy-nginx
    restart: always
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./volumes/conf.d:/etc/nginx/conf.d:ro
      - ./volumes/vhost.d:/etc/nginx/vhost.d:ro
      - ./volumes/certs:/etc/nginx/certs:ro
    networks:
      - localhost

  docker-gen:
    image: jwilder/docker-gen:0.7.3
    container_name: ld-proxy-gen
    restart: always
    depends_on:
      - nginx
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
      - ./templates:/etc/docker-gen/templates:ro
      - ./volumes/conf.d:/etc/nginx/conf.d:rw
      - ./volumes/vhost.d:/etc/nginx/vhost.d:ro
      - ./volumes/certs:/etc/nginx/certs:ro
    networks:
      - localhost
    entrypoint: /usr/local/bin/docker-gen -notify-sighup ld-proxy-nginx -watch -wait 5s:30s /etc/docker-gen/templates/nginx.tmpl /etc/nginx/conf.d/default.conf

networks:
  localhost:
    external: true

[ekyna]

Same here.

[hustlahusky]

I just found workaround for this. I have moved custom configs from /etc/nginx/conf.d to other directory and include them in nginx.tmpl and simply use a named volume instead of FS mount. Error gone for now

[paradoxina]

Same problem with jwilder/docker-gen in https://github.com/evertramos/docker-compose-letsencrypt-nginx-proxy-companion win10

[paradoxina]

I do not have this problem with the old version Docker 2.1.5.0
2.1.7.0 has problem

[bathanh4996]

I have the same problem in windown 10

[strarsis]

@hustlahusky, @ekyna, @paradoxina, @bathanh4996:
There is actually a workaround. The underlying issue is that docker-gen somehow renames (moves) the file in a way that is incompatible with some Docker versions (notably recent Docker Desktop on Windows 10).

Instead of letting docker-gen write the generated file to a Docker-mounted directory,
let it write it to the internal container file system instead (e.g. just /tmp) where it works nicely and
then copy/move that file from there to the actual Docker-mounted destination using another program/command. cp/mv has no issues copying/moving into a Docker-mounted directory.
The -notify option is perfectly suited for this. Just let docker-gen invoke cp/mv after it updated the config file to copy/move it to the Docker-mounted destination. If you also run a restart/reload command, you can append it and use something like &&, the cp/mv command return after it copied the file so there should be no race condition.

Example docker-compose.yml for illustration:

version: '3.7'
services:
  ssh-gen:
    image: jwilder/docker-gen:0.7.0
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
      - ./templates:/etc/docker-gen/templates
      - ./output/:/output/

    # Workaround for https://github.com/jwilder/docker-gen/issues/317
    # Let docker-gen write to internal container file system,
    # then run cp as notify command to copy the config file to mounted Docker directory
    command: -only-published -watch -notify "cp /tmp/containers.conf /output/containers.conf && some-reload-command-if-you-need-it" /etc/docker-gen/templates/dev-containers.tmpl /tmp/containers.conf

In this example docker-gen writes the generated file to /tmp/containers.conf, inside the container file system, which works just fine, then it invokes cp to copy that file to a Docker-mounted destination. When cp was successful (&& operator), a reload/restart command can also be invoked as usual.

This workaround works for me with the latest Docker Desktop edge 2.2.2.0 (43066) on Windows 10 Pro x64 and WSL 1.

[strarsis]

@hustlahusky, @ekyna, @paradoxina, @bathanh4996, @matteocng:
In latest Docker Edge release (2.2.3.0 (43965)) the issue has been resolved now, no workaround needed.
Clarification: It works on WSL 1, no need to install WSL 2 for the fix.