Support relative paths with bind mounts (volumes)

[borekb]

As previously discussed in moby/moby#4830, it would be useful to be able to bind-mount relative paths like this:

docker run --mount type=bind,source=host-dir,target=/app ...

Note: The original issue suggests docker run -v host-dir:/app ... which might (probably?) be confused with named volumes so I think it's more likely to expect it from the newer --mount option.

The two main motivations for this are:

• Consistency with docker-compose.
• Ability to write cross-platform docker run commands, e.g., we have a npm script that currently does something like "start": "docker run -v $PWD:/app ...". It's not easy at all to write a command that would work across Linux, macOS and Windows today.

[JonasJR]

This would be verry helpful! Been looking for this but havent found any good solution....

[prankymat]

In my opinion, it is explicit and makes sense to resolve the path as relative path instead of absolute if the bind mount path starts with a ./.

For example:

docker run -v ./data-dump:/app/dump my-image

./ resolves as relative path in docker-compose as well which makes perfect sense and should be supported in docker cli as well imho.

[felschr]

Coming from docker-compose which supports relative paths for volumes I would've expected docker itself to support it.

[Downchuck]

Seems like this one got dropped on contributor agreement -- otherwise looks like an easy one to get through.

[prankymat]

Would highly appreciated if there’s any updates regarding this issue.

[Downchuck]

@prankymat looks like it failed the automated tests on a rw label

[gpapin]

You can always use replace . by $(pwd) to achieve the same result in the meantime Like so : docker run -v $(pwd)/data-dump:/app/dump my-image

[borekb]

@gpapin You can't, your example won't work in some shells on Windows. See the original post where this is mentioned.

[gpapin]

You are right @borekb, this is just a workaround for linux like you mentioned in your original post.

[prankymat]

Casual ping that I've addressed this in #1273 which just passed all the tests. Would be more than happy if any moderators could help review it! 😄

[FireEmerald]

As workaround for $(pwd), try instead -v %cd%\on\host:/opt/inside/container which should work under windows.

[Lewiscowles1986]

Coming from docker-compose which supports relative paths for volumes I would've expected docker itself to support it.

docker-compose does not support this on all OS's. Windows for example is a nightmare for relative mounting regardless of ${PWD} and other what I'd term "hacks"

I don't understand why as OSx is also running in a VM, no matter how lightweight, vagrant and all VM technologies support this. It's crazy that since it's valuation, docker hasn't really innovated much at all.

[thaJeztah]

Thanks for opening this ticket. I know we rejected this / similar proposals in the past, but in light of "No is temporary, Yes is forever", perhaps it's time to have another look at this, to see if we are comfortable changing this.

Let me try to explain the background behind this (and reasons why it was implemented as it is);

Reasons why this (so far) hasn't been supported;

First of all, due to the syntax used on the command-line:

• complexity with the --volume [<host-path>|<volume-name>:]<container-path> short-hand syntax;
  • syntax is used for both bind-mounts, named volumes, and anonymous volumes, which makes it ambiguous.
  • due to the above, relative paths can only be implemented "partially"; for example, ./some-dir:/some/container/path could be supported, but not some-dir:/some/container/path, as that could either be a bind-mount (some-dir) or a volume named some-dir.
• the --mount type=bind,src=<host-path> should not have this problem (as it should not have the ambiguity between "bind-mounts" and "volumes"

The second reason is more "fuzzy", and harder to describe, but may be good to call out explicitly, as there's parts that may need to be looked into (also in general);

Bind-mounts are performed on the daemon-side; when creating a container that uses a bind-mount, the <host-path> that is specified will be used on the daemon side, which may be on a different machine as the cli (and could even be a different platform, e.g. a docker CLI running on macOS or Windows, and the dockerd daemon running on Linux (or Windows). This has been a source of confusion for many (especially when using -v / --volume, which auto-creates missing host-paths); "mentally", users expect <host-path> to be a path on the machine where they're running the docker command. Docker Desktop (and boot2docker before that) added to this mental concept, by trying to make the host-VM boundary as transparent as possible when using bind-mounts.

So how does that relate to this proposal? Well, it's somewhat orthogonal (bear with me); as mentioned, bind-mounts happen on the daemon side. For example, if I specify -v /home/MY_USERNAME/my-project:/app, I (implicitly) make the assumption that there's a MY_USERNAME user-account on the daemon side. This MAY be true (a local setup), but often is not the case (e.g. in situations where the daemon runs somewhere in the cloud). The docker CLI cannot make any assumptions whatsoever about paths on the daemon side, as it doesn't have that information.

For this, reason, we (so far) did not implement logic in the CLI that would suggest otherwise; the CLI parses the argument, and delegates any other validation to the daemon (which can check if the path is valid, and if it exists). The path sent to the daemon must be an absolute path (otherwise it would be "relative to dockerd"), and it's left as the user's responsibility to provide that path (which could involve shell mechanisms, such as $(pwd), ~, or $HOME).

So, is it worth continue to enforce absolute paths, or should we be fine with accepting relative paths?

As outlined above, there were good reasons to require absolute paths. Question is now; should we continue to do this? Yes; we still cannot make any assumption about paths on the daemon side. And, yes, this situation still holds for any "remote daemon" where no special handling (as provided by, e.g., Docker Desktop) is in place. However, for "inner loop" / "local" development, the current situation may be causing more confusion than it helps preventing. I expect the "this path must be absolute, because it's a remote path" message to be lost on most users.

Looking at some other commands / options that do accept relative paths;

• docker build <path to context>, docker build -f <path to Dockerfile>, docker build --output..., docker build --iidfile ...
• docker run / docker create --cidfile, --env-file, --label-file, --security-opt seccomp-profile=...
• docker cp <container-path> <host-path>
• docker export --output and docker import
• docker-compose (compose-file) works with paths relative to the location of the compose file

I should note that most of these, with the exception of the Docker Compose case, act on local file-paths, not remote/daemon side, but this distinction is hard to explain to users. Docker Compose originated from being a "developer tool", and always had a stronger preference for "local development" (against a local daemon, not a "remote" deployment).

What to do with other "special" paths?

Somewhat related to this discussion; handling of ~. Docker itself does not expand ~ to "home-dir"; expansion of this is handled by the shell. I've seen confusion around this, especially in situations where someone writes an example using ~, which works on macOS and Linux, but may fail on Windows (resulting in it being used as a literal ~). I don't think we should try to expand these, but perhaps we should consider printing warnings if there's a (likely) mistake. For example, if the shell didn't expand ~, and the docker CLI receives this character, we could print a warning. Perhaps even do the same for other cases, such as when we find a literal $HOME, %HOMEPATH%, or $(pwd).

How to handle conversion for non-matching platforms?

This is even more orthogonal, so just a brief mention; perhaps we need to look if we can make the path-conversion for non-matching platforms "pluggable". Currently, Docker Desktop contains logic to convert (e.g.) a Windows C:\some\directory path to corresponding paths inside the Docker Desktop VM (e.g. /mounts/host/some/directory). Would be good to have this logic handled in a more "standardised" way; but let's keep that for a separate discussion.

[thaJeztah]

From the above, I (personally) think it'd be ok to allow relative paths; with the constraints that relative paths MUST start with a period (.) and either followed by a path separator (/ or .\\), or a colon (for the -v shorthand syntax); that way, relative paths can be used both for the --mount and for the -v / --volume short-hand syntax (without the ambiguity between some-dir being a volume name or a directory named some-dir).

I guess technically we could implement some smart logic to handle relative paths without a leading ./ by first checking if some-dir exists in the current directory, and if so, expand it to the full path, otherwise treat it as volume-name, but I'm not too keen on that approach; this may lead to unexpected results, similar to MinGW and MSYS trying to be smart (which caused a lot of grieve).

Let me ping some other people who may have thoughts/opinions on this; @tianon @cpuguy83 @tonistiigi @AkihiroSuda @silvin-lubecki @justincormack @crazy-max

[cpuguy83]

I'm totally fine with it, compose already does this.
Probably will be good to take extra care in the docs to point out that relative such paths need to exist on the host machine but the client will be the one resolving the path.