docker cp to and from containers

[jlhawn]

Copy files/folders between containers and the local filesystem.

Usage:  docker cp [options] CONTAINER:PATH LOCALPATH|-
        docker cp [options] LOCALPATH|- CONTAINER:PATH

--help  Print usage statement

In the first synopsis form, the docker cp utility copies the contents of
PATH from the filesystem of CONTAINER to the LOCALPATH (or stream as
a Tar Archive to STDOUT if - is specified).

In the second synopsis form, the contents of LOCALPATH (or a Tar Archive
streamed from STDIN if - is specified) are copied from the local machine to
PATH in the filesystem of CONTAINER.

You can copy to or from either a running or stopped container. The PATH can
be a file or directory. The docker cp command assumes all CONTAINER:PATH
values are relative to the / (root) directory of the container. This means
supplying the initial forward slash is optional; The command sees
compassionate_darwin:/tmp/foo/myfile.txt and
compassionate_darwin:tmp/foo/myfile.txt as identical. If a LOCALPATH value
is not absolute, is it considered relative to the current working directory.

Behavior is similar to the common Unix utility cp -a in that directories are
copied recursively and file mode, permission, and ownership are preserved if
possible.

Assuming a path separator of /, a first argument of SRC_PATH and second
argument of DST_PATH, the behavior is as follows:

• SRC_PATH specifies a file
  • DST_PATH does not exist
    • the file is saved to a file created at DST_PATH
  • DST_PATH does not exist and ends with /
    • Error condition: the destination directory must exist.
  • DST_PATH exists and is a file
    • the destination is overwritten with the contents of the source file
  • DST_PATH exists and is a directory
    • the file is copied into this directory using the basename from
      SRC_PATH
• SRC_PATH specifies a directory
  • DST_PATH does not exist
    • DST_PATH is created as a directory and the contents of the source
      directory are copied into this directory
  • DST_PATH exists and is a file
    • Error condition: cannot copy a directory to a file
  • DST_PATH exists and is a directory
    • SRC_PATH does not end with /.
      • the source directory is copied into this directory
    • SRC_PAPTH does end with /.
      • the content of the source directory is copied into this
        directory

The command requires SRC_PATH and DST_PATH to exist according to the above
rules. If SRC_PATH is a symbolic link, the symbolic link, not the target, is
copied. If a path separator immediately follows the symbolic link, it will be
resolved to its target and the target resource will be copied.

A colon (:) is used as a delimiter between CONTAINER and PATH, but :
could also be in a valid LOCALPATH, like file:name.txt. This ambiguity is
resolved by requiring a LOCALPATH with a : to be made explicit with a
relative or absolute path, for example:

`/path/to/file:name.txt` or `./file:name.txt`

It is not possible to copy certain system files such as resources under
/proc, /sys, /dev, and mounts created by the user in the container.

Using - as the first argument in place of a LOCALPATH will stream the
contents of STDIN as a Tar Archive which will be extracted to the PATH in
the filesystem of the destination container. In this case, PATH must specify
a directory.

Using - as the second argument in place of a LOCALPATH will stream the
contents of the resource from the source container as a Tar Archive to
STDOUT.

• replaces docker cp to, from, and between containers #10198
• fixes Moving a file from the host system to a container via docker cp #5846
• fixes docker cp should accept a file destination #4272
• fixes docker cp does not cause a conflict when the archived directory structure replaces a directory with a file or vice versa #10040

[vbatts]

Oh my goodness. I wanted this some much last year, but we couldn't agree on the syntax of addresssing src and dest.

[duglin]

Not sure if it impacts your help comment, but FYI: #11858

[jlhawn]

@duglin I would definitely have to rebase if that gets merged before this ;-)

[icecrime]

Thanks for the work and for the wait @jlhawn, and thanks to all reviewers who participated in this journey.

[jlhawn]

🎉 Thanks everyone!

[tobegit3hub]

Great job @jlhawn 👍

[sa2ajj]

😄

[thaJeztah]

Wow, it's there!! looks like you need to update your story #13171 (comment) @jlhawn :-)

[tonistiigi]

Some follow-up...

Symlink sources don't seem to rebase properly:

docker run --name t0 busybox sh -c "mkdir /foo && touch /foo/bar && ln -s /foo /baz"
docker cp t0:/baz bax
ls -l # should have bax/ but has foo/

It's because client expects base directory baz in tar when it does the replacement but daemon returns foo.

---

If I run docker cp id:/foo bar I would expect that the client protects me against anything being put outside local file/dir bar. Currently client trusts API to provide valid data in this case. If there is anything weird inside the tar then it can leak out at least one level.

---

How can we protect this from running containers updating a filesystem/volumes when the request is taking place? I mean we check for breakouts in the beginning of GET/PUT but if a container is using the filesystem it could just flip a symlink in the right time and then we would have full read/write to host.

[tonistiigi]

Is this correct? GetResourcePath() should never return a symlink so I don't think this has much effect. Similar logic in ExtractToDir and comment in ArchivePath.

When I request stat for a symlink (with or without slash) I always get a directory as a response. AFAIK there isn't actually any harm of runnning Lstat on a path that is only joined and symlinks aren't evaluated. Reading/writing is different of course.

[jlhawn]

You're right, if FollowSymlinkInScope resolves all symlinks then that part doesn't matter. But, (It doesn't mention it in the comment), a trailing separator is also important because it asserts that the resource is a directory. The Lstat a couple of lines below this should capture that error condition (not a directory).

When I request stat for a symlink (with or without slash) I always get a directory as a response.

Is that when you stat a symlink on your local system or using this API?

Stat-ing a symlink with a trailing separator has different behavior depending on the system you're running on. Apparently on darwin, if a symlink foo points to a file bar and you call stat foo/ it will return stat info for bar even though bar is not a directory. On linux though, it will say stat: cannot stat 'foo/': Not a directory which is the error I expect it to pick up here.

[onesuper]

Great job @jlhawn

I have marked this serious and constructive discussion : )

[calebebrim]

I have tried something like this:

$ docker cp 0converted sleepy_rosalind:/home/test/data/aero_spectrum

cannot copy directory

What is wrong?
I'm running version 1.9.

where 0converted is an directory and aero_spectrum is another directory inside my container.

[stevvooe]

@calebebrim Please avoid commenting on closed issues. There are many other avenues to get support on using docker cp, such as the mailing list or IRC. If you still can't find a solution to your problem, you may then want to open an issue at that time.