Set owner of items created by Docker on the host

[AJGranowski]

Feature Request

I want to set the owner of directories and files created by Docker on the host from overlapping bind and named volumes.

Use case:
Using named volumes to increase performance on Windows without breaking permissions on Linux.

...
    cap_drop: ["ALL"]
    security_opt: ["no-new-privileges:true"]

    volumes:
      - type: "bind"
        source: "./"
        target: "/app"
      - type: "volume"
        source: "node_modules"
        target: "/app/node_modules"

    working_dir: "/app"

volumes:
  node_modules:

Possible implementations:

• Automatically set the running user as the owner for anything that Docker creates on the host.
• Add a UID:GID option to volumes, and have Docker chown anything that it creates on the host with that value.
• Add an option to compose config that prints the host paths that will be created by Docker, allowing a shell script to create them beforehand so they're not owned by root.

The problem I want to solve

If Docker creates a file or directory on the host system, it seems like those items are always owned by root. Here's a demonstration:

If we mkdir volume on the host before running Docker, volume will retain its owner. But if we don't, Docker will create it and it will be owned by root on the host.

======================================================================================================
--bind true --make-host-directory true --service nothing

Host:
uid=1000(user) gid=1000(user) groups=1000(user),27(sudo),997(docker),999(vboxsf)
drwxrwxr-x 2 user user 4096 Sep 27 21:46 volume

Container:
uid=1000(user) gid=1000(user) groups=1000(user)
drwxr-xr-x    2 root     root          4096 Sep 28 04:46 volume

Host:
uid=1000(user) gid=1000(user) groups=1000(user),27(sudo),997(docker),999(vboxsf)
drwxrwxr-x 2 user user 4096 Sep 27 21:46 volume
======================================================================================================
--bind true --make-host-directory false --service nothing

Container:
uid=1000(user) gid=1000(user) groups=1000(user)
drwxr-xr-x    2 root     root          4096 Sep 28 04:46 volume

Host:
uid=1000(user) gid=1000(user) groups=1000(user),27(sudo),997(docker),999(vboxsf)
drwxr-xr-x 2 root root 4096 Sep 27 21:46 volume

Setting the owner of the volume path in the image doesn't fix this issue, the directory must be created on the host to have a non-root host owner.

======================================================================================================
--bind true --make-host-directory false --service mkdir-chown

Container:
uid=1000(user) gid=1000(user) groups=1000(user)
drwxr-xr-x    2 user     root          4096 Sep 28 01:29 volume

Host:
uid=1000(user) gid=1000(user) groups=1000(user),27(sudo),997(docker),999(vboxsf)
drwxr-xr-x 2 root root 4096 Sep 27 21:46 volume
======================================================================================================
--bind true --make-host-directory true --service mkdir-chown

Host:
uid=1000(user) gid=1000(user) groups=1000(user),27(sudo),997(docker),999(vboxsf)
drwxrwxr-x 2 user user 4096 Sep 27 21:46 volume

Container:
uid=1000(user) gid=1000(user) groups=1000(user)
drwxr-xr-x    2 user     root          4096 Sep 28 01:29 volume

Host:
uid=1000(user) gid=1000(user) groups=1000(user),27(sudo),997(docker),999(vboxsf)
drwxrwxr-x 2 user user 4096 Sep 27 21:46 volume

This issue doesn't appear in WSL, but that difference is likely caused by WSL instead of Docker.

Why should this feature be added to Docker?

It seems unreasonable to ask users to create directories before Docker just so they're not owned by root. Ownership on the host seems like something that should be configurable, rather than a fight against Docker to achieve.

---

Docker version/info

Client: Docker Engine - Community
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:31:44 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:31:44 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.24
  GitCommit:        61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
 runc:
  Version:          1.1.9
  GitCommit:        v1.1.9-0-gccaecfc
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0



Client: Docker Engine - Community
 Version:    24.0.6
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 15
 Server Version: 24.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
 runc version: v1.1.9-0-gccaecfc
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.2.0-33-generic
 Operating System: Ubuntu 22.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 3.816GiB
 Name: Ubuntu
 ID: 74a82b1a-1a5d-465f-b6ae-4d3caac69c39
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Related issues/keywords:

npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /app/node_modules
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, mkdir '/app/node_modules'
npm ERR!  [Error: EACCES: permission denied, mkdir '/app/node_modules'] {
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'mkdir',
npm ERR!   path: '/app/node_modules'
npm ERR! }
npm ERR! 
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR! 
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator.

npm ERR! code EACCES
npm ERR! syscall open
npm ERR! path .../.npm/_cacache/_lastverified
npm ERR! errno -13
npm ERR! 
npm ERR! Your cache folder contains root-owned files, due to a bug in
npm ERR! previous versions of npm which has since been addressed.
npm ERR! 
npm ERR! To permanently fix this problem, please run:
npm ERR!   sudo chown -R 1001:1001 ".../.npm/"

[AJGranowski]

Rootless mode appears to address this issue:

======================================================================================================
--bind true --make-host-directory false --service nothing

Container:
uid=1000(user) gid=1000(user) groups=1000(user)
drwxr-xr-x    2 root     root          4096 Sep 29 00:35 volume

Host:
uid=1000(user) gid=1000(user) groups=1000(user),27(sudo),997(docker),999(vboxsf)
drwxr-xr-x 2 user user 4096 Sep 28 17:35 volume

Unfortunately many CI runners use rootful Docker.

[thaJeztah]

When bind-mounting a directory, files "inside" and "outside" the container are the same (I should mention here: on a Linux machine; the situation is different on Docker Desktop). This means that permissions of files and their ownership are exactly the same. On Linux, those permissions are based on user and group ID (numeric value); user and group names are effectively just "presentation" (and what names are shown, depends on what's present on the host if you look from the host).

So, in the case of a bind-mount, docker itself is not in the middle of things; when you specify a bind-mount ("host-path" to be mounted at "container-path"), that configuration is passed to the OCI runtime, which mounts the host-path into the container's mount-namespace. After that, processes
running inside the container can write to that path, and permissions as well as ownership is based on the user/group those processes are running as. If the process runs as root inside the container; files will be owned by root.

If you want those files to have different permissions, there are some options;

• start the container as a different user (docker run --user=123:456 .....), which may work for some cases, but not all (i.e., if the process running inside the container requires root, it may not be able to have the right privileges inside the container); of course, if the process inside the container is root, it's possible for it to chown files that were created to have the ownership of the user on the host.
• configure the daemon to be running with user-namespaces enabled; in this case user- and group-id's can be "mapped" (non-privileged user on the host equals "privileged" user inside the container)
• run the daemon in rootless mode; this uses similar techniques under the hood (but now the daemon as a whole runs without privileges).

Depending on your situation, you can also run Docker Desktop (for Mac, Windows, or Linux); Docker Desktop runs the daemon (and containers) inside a VM; when bind-mounting files from your host, those files are mapped into the VM, and Docker Desktop allows permissions to be mapped; this does, however, come with an overhead, so may be less suitable for heavy data read/writes.

For some use-cases, the --output option on docker build can be useful; with that feature it's possible to run a docker build and, instead of producing an image, export the files from a stage in you Dockerfile to the host; that stage in the Dockerfile can be set up to change permissions / ownership before exporting (which could allow you to map permissions); https://docs.docker.com/engine/reference/commandline/buildx_build/#output

Longer term, we anticipate providing support for "id-mapped mounts"; this is a feature in recent Linux kernel versions that allows mapping user-id's on a per mount base; this is similar to running containers with user-namespaces (user-mapping) and rootless, but per mount. With this, it's possible for a process inside the container running as (e.g.) root to write files as root inside the container, but to be "non-root" on the host.

Support for this has not yet been implemented, and (as mentioned) requires recent kernel versions, so if implemented, can't be supported on all Linux distros (running older kernel versions).

More discussion on this topic can be found in this ticket;

• Add ability to mount volume as user other than root moby/moby#2259

Some information can also be found in this answer I once wrote; https://stackoverflow.com/a/29251160/1811501 (but most of that I also captured above).

Given that this is not something that relates to the Docker CLI (as mentioned this all depends on the daemon-side), and we're tracking this features in moby/moby#2259, I'll close the ticket here, but feel free to continue the conversation.