New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to docker-compose up --force-recreate
without reusing previous volumes
#2127
Comments
We could probably document this a little better. |
This policy seems to violate docker best practices, and has caused us tons of headache troubleshooting why recreating containers was leaking state. When I tell docker to recreate, that doesn't mean "persist some data across the container runs but restart the processes in the containers". It means pave the earth and start over. If I wanted to save the volumes I would explicitly mount volumes. I never expect auto-mounted volumes of any kind to persist across container runs. |
Persisting data across runs is really the only reason to use volumes. If you don't want persisted data why are you putting it in a volume? |
I'm not setting up a volume. My Command:
docker-compose.yml:
Dockerfile:
create_test_db.sh:
If I run the above, write some stuff to the DB, then SIG_INT, then run the command again the data I put in the DB is persisted across the run. |
That's an issue with the MySQL image. It creates a volume in the base On Oct 19, 2016 6:36 PM, "Micah Zoltu" notifications@github.com wrote:
|
Hmm, this violates my understanding of docker containers. How is it mounting a volume without me providing a path on the host? My understanding is that docker containers are ephemeral unless you explicitly mount a volume? |
A volume doesn't need a host path. There are three kinds of volumes:
The mysql image uses an anonymous volume. At runtime you can tell the container to use a different volume for that path in the container, but if you don't the anonymous volume is still there. Anonymous volumes are not great. They are the oldest of the three, and much of their behaviour is legacy stuff that is only kept for backwards compatibility reasons. |
At least some command line option a la "--recreate-volumes" would be nice... |
OK, guys, here's the case to think of:
Currently, if I recreate containers, I end up with old version of "public" -- contents in the image is simply ignored. Yes, Docker is supposed to copy missing files into the anonymous volume, but for variety of reasons it does not always happen (I suspect, there's no deep check of subdirectories structure). So, I am either forced to perform "stop-rm-up" sequence (not that convenient in production), or to use separate directory as a shared volume, and explicitly call 'rsync' on application container start to populate/update it. If there was a way to just let anonymous volume go along with the parent container, it would be great improvement. |
Don't use volumes for code (or static assets). Volumes are for data that you want to preserve between deployments, which is the opposite of what you want here. Either build the nginx image with the static assets, or proxy some web server container that contains them. |
Thanks for the viewpoint! I did not thought from this perspective yet. It seems to be one of conceptual problems with (still evolving) Docker architecture. We saw evolution of data volumes concept so far (e.g. from "data containers" to "named volumes") and probably it is not yet finished. If you'll look at https://docs.docker.com/engine/tutorials/dockervolumes/#/data-volumes , you'll see that most described benefits (bypassing AUFS, sharing) are not necessarily connected to data persistence (which volumes were originaly designed for). So, no surprise people (including me) are trying to use volumes in a variety of ways beyond their original purpose. E.g. to share ephemeral or image-controlled data from one container to another, without lots of explicit copying. Maybe some day we'll figure out the consistent standard way to do it. :) So far, it is possible to use some fairly simple workarounds described above. Not ideal, but acceptable, as long as architectural expectations are properly set. Once again, thanks for your response, clarification and for the great work you are doing! |
I ran into this issue today, thanks for the great explanations here everyone, has definitely helped me understand the problem. A major point of confusion for me was understanding that the |
As mentioned @hleb-rubanau. Running single container with Rails and Nginx is the solution? (Like this: https://docs.docker.com/engine/admin/multi-service_container/ ?) Should I break the best practices ("Each container should have only one concern |
For those interested, I ended up with the following:
|
Thanks @hleb-rubanau Anyway, I'm not sure if some orchestration tools like RancherOS supports those volume configurations or is the right way in order to scale. At the end, is more difficult deploy with docker... I prefer to use anonymous volumes and clean the orphans at some point. This is my production rails stack: https://github.com/brunocascio/AR-MTB/blob/master/docker-compose.prod.yml |
This also caused me a large amount of confusion when I was testing the jenkins/jenkins image and it wasn't respecting my changes from the files in /usr/share/jenkins/ref because it had already copied them. That is a pretty unexpected user experience - docker compose is creating a "hidden" volume for all intents and purposes. If you use normal At the very least we should print a message saying "not re-creating volume x" so that other people in the future don't have to waste time wondering what is happening. |
Hi @dnephin, I call Getting this error
As you may have some idea to get around that, please share. Thank you! |
Just re-creating the folder will fix my issue. |
Big thanks to @dnephin, your answer worked perfectly, the first time. Muchos gracias. |
@dnephin said:
to share files between containers. Is there a better way to do that without using volumes? |
What do you mean by "share files"? Are you expecting that one container will write to a file, and the other container will see those writes? A file system is generally not a good interface between two services, but if that is the case you could have one of the services "manage" the filesystem by writing updates to the volume. If "sharing" is just that two containers happen to read some of the same files then there is no need for a volume. Add the file to both containers with |
I'm using docker-compose up -d --build --force-recreate --renew-anon-volumes db It seems the flag |
Thanks for letting this option. |
With docker-compose 1.4.2 and docker 1.8.2
You can see below the first volume e583c6a8...5a93788a0 is reused
I'm forced to
stop
thenrm
to create some new volumes_
Is there a better way than the
stop/rm
method?The text was updated successfully, but these errors were encountered: