Allow docker cp to copy host->container, container->host, container->container #6580

vieux · 2014-06-20T22:52:49Z

This allows:

docker cp <container_id>:/path path
docker cp path <container_id>:/path
docker cp <container_id>:/path <container2_id>:/path

With container to container, to keep the code simple (for now?) the content goes through the client's fs.

Needs API bump & doc

This one needs a lot of testing @crosbymichael @tiborvass @unclejack @LK4D4

crosbymichael · 2014-06-20T23:42:41Z

api/server/server.go

+		return fmt.Errorf("Missing parameter")
+	}
+
+	resource := r.Header.Get("Resource")


Is it ok to use the header for this ?

I'm not sure, do you have another idea ?

in the url?

cyphar · 2014-06-21T03:39:07Z

Wouldn't it be neater (and cleaner) to do this with interfaces? You take the two arguments and convert them to types which fulfill interfaces that allow you to copy to and from them.

LK4D4 · 2014-06-23T05:26:11Z

I'm not very sure how it must working :) But for now it is very strange behavior as for me:

$docker run --name cp_test -d ubuntu ping google.com
02e3e9395b802e241ecf17163aa13c0b08036d47b9b5fca564772fc022a890ba

$docker cp kernel_config cp_test:/.config

$file /var/lib/docker/btrfs/subvolumes/02e3e9395b802e241ecf17163aa13c0b08036d47b9b5fca564772fc022a890ba/.config
/var/lib/docker/btrfs/subvolumes/02e3e9395b802e241ecf17163aa13c0b08036d47b9b5fca564772fc022a890ba/.config: directory

$ls /var/lib/docker/btrfs/subvolumes/02e3e9395b802e241ecf17163aa13c0b08036d47b9b5fca564772fc022a890ba/.config

$docker cp cp_test:/etc/passwd ./pswd

$file ./pswd
./pswd: directory

$ls ./pswd  
passwd

Also

docker cp kernel_config cp_test:/

does nothing for me :(

LK4D4 · 2014-06-23T05:27:29Z

api/client/commands.go

+		pathSrc  = cmd.Arg(0)
+		pathDst  = cmd.Arg(1)
+	)
+	if len(infoSrc) != 2 && len(infoDst) != 2 {
 		return fmt.Errorf("Error: Path not specified")


I think it should be sort of "Wrong path format" and usage print.

LK4D4 · 2014-06-23T05:39:25Z

Also for from cont to cont works better:

$ docker cp cp_test:/etc/passwd cp_test2:/passwd
$ docker diff cp_test2
A /passwd
A /passwd/passwd

And there is no check about files existence on host.

LK4D4 · 2014-06-23T05:46:12Z

server/server.go

-		}
-		defer data.Close()
+		switch method {
+		case "OUT":


Maybe we should have two separate jobs for this cases?

LK4D4 · 2014-06-23T05:48:43Z

Also word copy in method names was pretty confusing for me when I read code. Maybe we should use sort of Read/WriteResource and same for jobs. Don't know if anyone have same feeling, and maybe someone could propose better naming.

vieux · 2014-06-23T18:22:36Z

@LK4D4 "And there is no check about files existence on host." ?

LK4D4 · 2014-06-23T18:36:38Z

@vieux

$ls asd
ls: cannot access asd: No such file or directory

$docker cp asd cp_test2:/asd

vieux · 2014-06-23T22:32:28Z

@LK4D4 addressed most of the comments, regarding the failures I think we have an issue with files in the Archive package: https://github.com/vieux/docker/compare/dotcloud:master...vieux:tar_untar_file?expand=1

SvenDowideit · 2014-06-24T02:32:35Z

neat! a pony, and in my favourite colour too!!

tiborvass · 2014-06-24T13:58:19Z

@vieux remove "file" from the path.Join in both the Tar and Untar function calls.

LK4D4 · 2014-06-24T18:13:22Z

Also, we should check autocomplete. Doesn't work in zsh for me at least.

vieux · 2014-06-24T18:55:06Z

@LK4D4 feel free to submit a PR to my branch to update all the completion scripts..:)

tianon · 2014-06-25T01:19:41Z

Autocomplete here is already kind of annoying since we don't have something
like "docker ls".

LK4D4 · 2014-06-25T04:08:26Z

@tianon ah, also there was two PRs with ls IIRC, but @crosbymichael rejected them and maybe he was right :)
I want at least host-side completion.

vieux · 2014-06-25T20:22:24Z

Isn't completion handled by the bash/zsh/fish scripts ?

tianon · 2014-06-25T20:27:46Z

Yeah, I can PR an update to bash, but I'm not zsh or fish fluent so can't help much there.

tianon · 2014-06-25T20:46:03Z

vieux#6

vieux · 2014-06-26T22:45:47Z

integration-cli/docker_cli_cp_test.go

+		errorOut(err, t, fmt.Sprintf("failed to cp to the container: %v", err))
+	}
+
+	/* TODO: allow overriding files


ping @tiborvass

vieux · 2014-06-26T22:47:02Z

I updated. 2 things for @tiborvass .

I think we should be able to override a file
docker cp file <container_id>:/ doesn't work as you tweaked this earlier this week, can you please take a look ?

Docker-DCO-1.1-Signed-off-by: Victor Vieux <victor.vieux@docker.com> (github: vieux)

Docker-DCO-1.1-Signed-off-by: Victor Vieux <vieux@docker.com> (github: vieux)

Docker-DCO-1.1-Signed-off-by: Tibor Vass <teabee89@gmail.com> (github: tiborvass)

Fix cp host container to root

vieux · 2014-06-28T00:10:29Z

@LK4D4 can you retry now ?

LK4D4 · 2014-06-28T07:37:14Z

@vieux yup, I'll try on weekend

LK4D4 · 2014-06-29T07:21:14Z

moroz@minigrind ~ % docker cp copier1:/urlview copier2:/shuview                                                                         
moroz@minigrind ~ % docker cp copier1:/urlview copier2:/shuview
2014/06/29 11:17:11 Error: Trying to untar an archive with multiple entries to an inexistant target `/var/lib/docker/btrfs/subvolumes/81dd7864831d4945ba99bbd58d07f5e6c36cc8c87e5c7b503104cb9ce8642c31/shuview`: did you mean `/var/lib/docker/btrfs/subvolumes/81dd7864831d4945ba99bbd58d07f5e6c36cc8c87e5c7b503104cb9ce8642c31` instead?

After first command shuview is a file, after second - directory with file urlview, all following cp leave it as a directory.

Docker-DCO-1.1-Signed-off-by: Andrew Page <admwiggin@gmail.com> (github: tianon)

Update bash-completion for bidirectional "docker cp"

proppy · 2014-07-02T18:08:29Z

api/server/server.go

@@ -994,6 +994,28 @@ func postContainersCopy(eng *engine.Engine, version version.Version, w http.Resp
 	return nil
 }

+func putContainersCopy(eng *engine.Engine, version version.Version, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
+	if vars == nil {
+		return fmt.Errorf("Missing parameter")


maybe be more explicit in the error message about which parameter you're expecting.

cpuguy83 · 2014-07-03T16:05:32Z

This is sweet.

… multiple entries in the archive stream Docker-DCO-1.1-Signed-off-by: Tibor Vass <teabee89@gmail.com> (github: tiborvass)

vieux · 2014-07-08T23:29:59Z

Closing for now, will reopen when we figure out the tar stuff

duglin · 2014-08-12T23:50:28Z

Just curious, if windows is ever supported would things like:
docker cp container_id:/path container2_id:/path
cause an ambiguity problems? Granted container_id is usually more than just one character, but its not a requirement.

duglin · 2014-08-13T13:26:41Z

Ignoring the client command line syntax for a moment, and just focusing on the REST API, I was wondering what people thought about cleaning it up a bit? As I mentioned in https://groups.google.com/forum/#!topic/docker-dev/TmaCiWIEyVk using POST to retrieve a file feels a bit odd and having the path to the file be in the http body is a bit limiting.

I was wondering what people thought about adding new APIs that were a bit more natural/RESTy, in particular:
GET /containers/{id}/resources/{path} to copy a file from a container to the host
HEAD /containers/{id}/resources/{path} to check for existence of a file in a container (or get just metadata)
PUT /containers/{id}/resources/{path} to copy a file from the host to the container - where the contents of the file are either:
a) in the http body, or
b) referenced by an http header (e.g. DOCKER_SRC: {container-id}:{path} ) in the case of container-to-container copy
This could use POST instead I guess, but I'm more used to POST being used in cases where the target URL of the resource is generated by the server and PUT being used when the target URL is already known by the client.
DELETE /containers/{id}/resources/{path} to delete a file from the container - which seems only natural once you have the idea of creating a file at all.

I'd be interested in working on a PR on this but only if there's interest - aside from the container->container copy, I think most of the code is already there and its just minor edits to expose it slightly differently. The old APIs can stick around and be deprecated. Thoughts?

duglin · 2014-08-15T14:57:19Z

@vieux - was wondering if you had any feedback on this? I'm most interested in the support for checking to see if a file is present in a container - the other stuff around it, while interesting, isn't as critical to my needs right now. I'd really like to submit a PR for this "does a file exist" feature but don't want to code it up until I get a sense for the direction people would like to see it go. .

tobegit3hub · 2015-05-21T08:54:39Z

@duglin is right! But we also like the feature to copy the file from host to container. Maybe we don't need to copy from container to container because we can do it by the existing methods.

crosbymichael reviewed Jun 20, 2014
View reviewed changes

vieux added the /project/doc label Jun 23, 2014

LK4D4 reviewed Jun 23, 2014
View reviewed changes

server/server.go

}

defer data.Close()

switch method {

case "OUT":

Copy link

Contributor

LK4D4 Jun 23, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should have two separate jobs for this cases?

tiborvass mentioned this pull request Jun 25, 2014

Improve untar when using files instead of directories. Specifies behavior on non-existant targets. #6661

Merged

vieux reviewed Jun 26, 2014
View reviewed changes

vieux mentioned this pull request Jun 27, 2014

Allow overwrite in archive.Untar #6740

Merged

vieux added 3 commits June 27, 2014 23:44

add test

880c974

Docker-DCO-1.1-Signed-off-by: Victor Vieux <victor.vieux@docker.com> (github: vieux)

add scp style

e80fef8

Docker-DCO-1.1-Signed-off-by: Victor Vieux <victor.vieux@docker.com> (github: vieux)

add basic support for container to container

3820e2e

Docker-DCO-1.1-Signed-off-by: Victor Vieux <victor.vieux@docker.com> (github: vieux)

vieux and others added 9 commits June 27, 2014 23:44

update wording

3ae0d8e

Docker-DCO-1.1-Signed-off-by: Victor Vieux <vieux@docker.com> (github: vieux)

use two jobs

aec2717

Docker-DCO-1.1-Signed-off-by: Victor Vieux <vieux@docker.com> (github: vieux)

use URL

b98bad6

Docker-DCO-1.1-Signed-off-by: Victor Vieux <vieux@docker.com> (github: vieux)

fix use of getResourcePath

35d1e03

Docker-DCO-1.1-Signed-off-by: Victor Vieux <vieux@docker.com> (github: vieux)

add stat before cp

f461b3f

Docker-DCO-1.1-Signed-off-by: Victor Vieux <vieux@docker.com> (github: vieux)

update test

802efc7

Docker-DCO-1.1-Signed-off-by: Victor Vieux <vieux@docker.com> (github: vieux)

allow file

a4592dc

Docker-DCO-1.1-Signed-off-by: Victor Vieux <vieux@docker.com> (github: vieux)

fix docker cp file container:/

9aaa8e8

Docker-DCO-1.1-Signed-off-by: Tibor Vass <teabee89@gmail.com> (github: tiborvass)

Merge pull request #7 from tiborvass/fix-cp-host-container-to-root

f0649c9

Fix cp host container to root

tianon and others added 2 commits June 30, 2014 18:32

Update bash-completion for bidirectional "docker cp"

a4b0eb9

Docker-DCO-1.1-Signed-off-by: Andrew Page <admwiggin@gmail.com> (github: tianon)

Merge pull request #8 from tianon/scp-bash-completion

2b4fb49

Update bash-completion for bidirectional "docker cp"

proppy reviewed Jul 2, 2014
View reviewed changes

vieux mentioned this pull request Jul 2, 2014

Moving a file from the host system to a container #905

Closed

vieux added this to the 1.1.1 milestone Jul 3, 2014

tiborvass referenced this pull request in tiborvass/docker Jul 8, 2014

Fix regression in Untar when destination does not exist and there are…

4611cb7

… multiple entries in the archive stream Docker-DCO-1.1-Signed-off-by: Tibor Vass <teabee89@gmail.com> (github: tiborvass)

vieux closed this Jul 8, 2014

vieux mentioned this pull request Aug 12, 2014

remove unused HostPath from /copy body #7543

Closed

vieux mentioned this pull request Aug 27, 2014

Proposal: support wildcards and destination names in docker cp #7710

Open

remram44 mentioned this pull request Sep 2, 2014

Moving a file from the host system to a container via docker cp #5846

Closed

jlhawn mentioned this pull request May 20, 2015

docker cp to and from containers #13171

Merged

thaJeztah mentioned this pull request Feb 9, 2021

Feature Request: Allow copying between containers #42006

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow docker cp to copy host->container, container->host, container->container #6580

Allow docker cp to copy host->container, container->host, container->container #6580

vieux commented Jun 20, 2014

crosbymichael Jun 20, 2014

vieux Jun 20, 2014

crosbymichael Jun 20, 2014

cyphar commented Jun 21, 2014

LK4D4 commented Jun 23, 2014

LK4D4 Jun 23, 2014

LK4D4 commented Jun 23, 2014

LK4D4 Jun 23, 2014

LK4D4 commented Jun 23, 2014

vieux commented Jun 23, 2014

LK4D4 commented Jun 23, 2014

vieux commented Jun 23, 2014

SvenDowideit commented Jun 24, 2014

tiborvass commented Jun 24, 2014

LK4D4 commented Jun 24, 2014

vieux commented Jun 24, 2014

tianon commented Jun 25, 2014

LK4D4 commented Jun 25, 2014

vieux commented Jun 25, 2014

tianon commented Jun 25, 2014

tianon commented Jun 25, 2014

vieux Jun 26, 2014

tiborvass Jun 27, 2014

vieux commented Jun 26, 2014

vieux commented Jun 28, 2014

LK4D4 commented Jun 28, 2014

LK4D4 commented Jun 29, 2014

proppy Jul 2, 2014

cpuguy83 commented Jul 3, 2014

vieux commented Jul 8, 2014

duglin commented Aug 12, 2014

duglin commented Aug 13, 2014

duglin commented Aug 15, 2014

tobegit3hub commented May 21, 2015

Allow docker cp to copy host->container, container->host, container->container #6580

Allow docker cp to copy host->container, container->host, container->container #6580

Conversation

vieux commented Jun 20, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cyphar commented Jun 21, 2014

LK4D4 commented Jun 23, 2014

Choose a reason for hiding this comment

LK4D4 commented Jun 23, 2014

Choose a reason for hiding this comment

LK4D4 commented Jun 23, 2014

vieux commented Jun 23, 2014

LK4D4 commented Jun 23, 2014

vieux commented Jun 23, 2014

SvenDowideit commented Jun 24, 2014

tiborvass commented Jun 24, 2014

LK4D4 commented Jun 24, 2014

vieux commented Jun 24, 2014

tianon commented Jun 25, 2014

LK4D4 commented Jun 25, 2014

vieux commented Jun 25, 2014

tianon commented Jun 25, 2014

tianon commented Jun 25, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vieux commented Jun 26, 2014

vieux commented Jun 28, 2014

LK4D4 commented Jun 28, 2014

LK4D4 commented Jun 29, 2014

Choose a reason for hiding this comment

cpuguy83 commented Jul 3, 2014

vieux commented Jul 8, 2014

duglin commented Aug 12, 2014

duglin commented Aug 13, 2014

duglin commented Aug 15, 2014

tobegit3hub commented May 21, 2015