-
Notifications
You must be signed in to change notification settings - Fork 2k
Proposal: machine share #179
Comments
@nathanleclaire thanks for the proposal -- well written :) A couple of initial thoughts: the idea of sharing I think makes sense for vbox (similar to what is recommended for fig) and we already do that for the vbox driver. However, outside of that, IMO they should be Docker images. I think it would be handy to be able to copy files (perhaps a Are there other use cases you could think of besides app code? To be clear, I'm not arguing against it, just trying to better understand the usage :) |
+1 for |
The basic reasoning that many users ave given is that when they are developing, they want the source code to be on their local machine, that way, when they blow away the remote instance, they don't have to worry about the files going away. I recon this concern will exist for those of us that will use ephemeral cloud Docker daemons just the same as for b2d style ones. so I'm very +1 to this - as its similar to what i started exploring in boot2docker/boot2docker-cli#247 |
Perhaps local drivers could implement an arg for this. But cloud providers shouldn't have this. |
scp and/or rsync would be good, same as current machine ssh. |
@sthulb can you elaborate on why not? this will be a common question, and so will need to be in the docs (its not that hard to envision an opportunistic |
Local machines are geared for dev environments. I feel like remote (cloud) machines are for production purposes and thus, should have their data provisioned with data containers, or via chef/puppet/ansible/etc. I can understand the requirement to upload a single file to a remote host though (various secrets). I'm willing to sway if others feel like this is a thing we should have. |
I have to agree with @sthulb. I think we should stay away from syncing/sharing in remote environments. I can see it being extremely powerful for local. |
I use digital ocean as a dev environment. I also have 2 non-vm physical boot2docker servers that I use for development and testing, where my sshfs share driver for b2d-cli is very useful. |
Perhaps this is a case for plugins. |
@sthulb i think that's a brilliant idea |
+1 - if you look at the PR I made in the b2d-cli repo, I copied the driver model, but starting machine with plugins would be nicer. |
+1 for scp, and I agree with comments above, this proposal looks useful for development machines (Vbox, fusion), but on the other providers I think this could make quite a mess real fast. |
+1 for scp, (perhaps sftp?) |
+1 for local VM shares, NFS is a portable option if driver-specific shares are awkward. Vagrant does this kind of thing well :) I'd love to see it in this project. |
+1 for scp |
To give interested parties an update on where my head is at with this one: I'd like to make a Then, for more complicated use case e.g. NFS/Samba, there would be a separate (still |
I'm starting to think we should have a git style interface, with executables starting docker-machine-foo, in this case foo would be share. |
This would be great for extending machine. |
Just tried docker-machine v0.1.0 for the first time today. I have a workflow involving live updates to a host directory being propagated to a mounted volume within a container. I'm using the standard docker CLI arguments to achieve this.
I'm very impressed with docker-machine so far. My project's unit tests (ones unrelated to the mounted volume) passed just as they used to with boot2docker. Besides this issue, this was an extremely seamless transition from boot2docker to docker-machine. |
@jokeyrhyme the issue with fusion is known and is being worked on. Thanks for the great feedback! |
+1 for scp |
@jcheroske It's in master now!! #1140 |
That's incredible. Noob question: Is there an osx binary of the latest and On Tue, May 19, 2015 at 1:05 PM, Nathan LeClaire notifications@github.com
|
@jcheroske You can always get the latest (master) binaries from here: https://docker-machine-builds.evanhazlett.com/latest/ :) |
Are there any updates on this? |
Windows, rsync and the fswatch problems: In order to reduce hooping through github issues, i reference explicitly: In a nutshell, for serious development you need the inotify (or corresponding) functionality to work. fswatch implements an intersting library, but which does not support Windows' FileSystemWatch API. A good starting point on this issue is https://github.com/thekid/inotify-win/, but: thekid/inotify-win#7 |
Docker is such a mess. I think is not good as it seems. |
We are not in Docker Compose support forum, so please don’t discuss it there. People are subscribed to this issue because of other topic. Read tutorial about Docker Compose on docker.com. It is great utility, you just using it wrong.
|
+1 +1 +1 +1 +1 +1 +1 +1 +1 +1 +1 +1 +1 +1 current virtualbox sharing is performance limited and has permissions issues with some containers (postgresql for example) |
+1 |
1 similar comment
+1 |
so the solution is what? what should i do in my mac?ヽ(゜Q。)ノ? |
Please, write there only constructive comments. No „+1“, because it is send If you want to support this issue, just subscribe it. Jan Pobořil 2016-01-11 5:03 GMT+01:00 daben1990 notifications@github.com:
|
Running this command works fine for copying between development machines on the same network: rsync -chavzP --stats other-computer.local:~/.docker/machine/ ~/.docker/machine/ --exclude machines/default --exclude cache |
+1 |
Finally I found the right issue, @nathanleclaire I would bet CIFS/NFS docker volumes, as implemented by https://github.com/gondor/docker-volume-netshare would solve the problem immediately. However those would need to be integrated seamlessly into boot2docker. Would that be possible? |
I think moby/moby#20262 can be an advent to the final answer to this. |
I use something along this line to keep in sync:
edit: typo |
Very interested in this feature as well, signing up for notifications. IMO - based on the thread, it sounds like the easiest and most useful implementation would be to use standard drivers in the image to create local-only volume mounts, and not implement the feature remotely (if you are - you're sort of doing it wrong, right?). Value for myself would come in local, on-the-fly development to contents in the container without any sftp / scp / rsync steps involved (or scripts like the one that MBuffenoir has set up) |
Same here. This is a must have. |
@in10se sorry to say but Docker Machine seems to be a dead product. Development ans innovation has stagnated and I wouldn’t count on it for production environments. I think Hashicorp Terraform or similar tools are the way to go. |
@erkie, it does look like activity has slowed down in the past few months. Hopefully it will resume. Thanks for the suggestion. |
What is the state of this proposal? |
Not much going on here. |
7 years???? |
machine share
Abstract
machine
is shaping up to be an excellent tool for quickly and painlessly creating and managing Docker hosts across a variety of virtualization platforms and hosting providers. For a variety of reasons, syncing files between the computer wheremachine
commands are executed and the machines themselves is desirable.Motivation
Containers are a great tool and unit of deployment and machine makes creating a place to run your containers easier than ever. There are, however, many reasons why one would want to share files which are not pre-baked into the Docker image from their
machine
client computer to a machine host. Some examples are:machine
host is VM on the user's laptop and they are developing a web application inside of a container. They want to develop with the source code bind-mounted inside of a container, but still edit on their computer using Sublime etc..csv
files up to the hosts from their laptop. This user does not know about the implementation details ofmachine
or how to find the SSH keys for the hosts etc.Like the rest of
machine
, it would be preferable to have 80% of use cases where this sort of thing happens integrated seamlessly into themachine
workflow, while still providing enough flexibility to users who fall into the 20% not covered by the most common use cases.Interface
After thinking about the mechanics and UX of this, I think we should favor explicitness over implicitness and err on the side of not creating accidental or implicit shares.
There are a few aspects to the story that should be considered.
Command Line
The syntax would be something like this:
IMO we should forbid users from creating shares to or from outside of the home directory of the client or the remote host. There's a strong argument that the home directory itself should be banned from sharing as well, to prevent accidental sharing of files which should be moved around carefully such as
~/.ssh
and, of course,~/.docker
. Also, clients could share directories to multiple locations, but any shares which point to the same destination on the remote would be disallowed.Totally open to feedback and changes on the UI, this is just what I've come up with so far.
Drivers
There is a huge variety of ways to get files from point A to point B and back, so I'd propose a standard interface that drivers have to implement to be recognized as an option for sharing (just like we have done with virtualization / cloud platforms). The default would be something like
scp
(since it is so simple and is pretty ubiquitous) and users would be able to manually specify one as well. Users could pick the driver that suits their needs. Additionally it would allow themachine
team to start with a simpler core and move forward later e.g. justrsync
,scp
, andvboxsf
could be the options in the v1.0 and then later other drivers could be added.Some possible drivers:
scp
,vboxsf
,fusionsf
,rsync
,sshfs
,nfs
,samba
This would be useful because different use cases call for different ways of moving files around. NFS might work well for development in a VM, but you might want
rsync
if you are pushing large files to the server frequently, and so on.Part of the interface for a share driver would be some sort of
IsContractFulfilled()
method which returns a boolean that indicates if the "contract" necessary for the share to work is fulfilled by both the client and remote host. This would allow us to, for instance, check ifrsync
is installed on the client machine and the remote host, and refuse to create a share if that is not the case. Likewise, it would prevent users from trying to do something silly like using thevboxsf
driver on a non-Virtualbox host.Possible Issues
machine
moves to a client-server model, which seems favored at the moment, it introduces additional complications to the implementation ofmachine share
. Namely,machine share
would be all client-side logic, and would not be able to make the same assumptions it may make today e.g. that the SSH keys for the hosts are all lying around ready to be used on the client computer.machine share push
andmachine share pull
make a ton of sense for some drivers, such asrsync
, but not so much sense forvboxsf
ornfs
which update automatically. What happens when a user doesmachine share push
on such a driver?"ERR: This driver is bidirectional"
?Other Notes
I am not attached to the proposal, I am just looking for feedback and to see if anyone else thinks this would be useful. I have put together some code which defines a tentative
ShareDriver
interface but it is not really to my liking (and no real sync functionality is implemented) so I haven't shared it.The text was updated successfully, but these errors were encountered: