docker-compose up doesn't pull down latest image if the image exists locally

[nrktkt]

It would be nice if there were an option to check for new versions of images when running docker-compose up.

We already have this functionality with docker build --pull as was discussed here moby/moby#4238 and there is an open issue to bring --pull to docker run here moby/moby#13331.

I propose adding --pull to up to always attempt to pull a newer version of the images in the compose file.

[shin-]

Is there a reason why docker-compose pull && docker-compose up is impractical?

[nrktkt]

I think most of the points for/against are the same as the ones discussed in the issue for adding --pull to docker run. They range from ux and consistency, to ease of scripting/workflow, to swarm integration (out of curiosity, what does docker-compose pull do with swarm?).
I don't think it's a major issue, but something to consider. The same type of users who desired the feature elsewhere would likely also enjoy it here.

[decoomanj]

I'm trying to execute "docker-compose build", but it doesn't refresh the image referenced in the Dockerfile, except when you use --pull.

You can also build containers during the start with up --build. But the newest images will not be pulled. Can we expect sthg like "docker-compose up --build --pull" (or similar)? Maybe it makes sense to place it in the YML since not all build must be refreshed (cfr. local images).

[cressie176]

Instead of (or in addition to) adding --pull to the cli, what about adding something on the service definition in the docker-compose file?, e.g.

version: '2'

services:

  postgres:
    image: postgres
    container_name: postgres
    pull: true
    ports:
     - '5432:5432'

This way if there's a service I don't care about being latest and one I do, docker-compose won't waste time downloading images I'm not interested in

[blockloop]

I came here looking for this feature because we use it in our production cluster of Kubernetes. There the tag is "imagePullPolicy" and it can be set to "IfNotPresent", "Always", or "Never." Something similar for a compose environment would be nice.

[PengBAI]

In our case, we need to rebuild base image everyday to make sure the latest dependencies updated for applications. Docker compose up to pull the latest image with the same tag is a nice feature. Why not !

[marcelovcpereira]

Any news about this?

[sugun999]

Hi,

Any news about this issue?

[jonathan-kosgei]

+1

[draketribe]

+1

[shin-]

As I mentioned earlier, docker-compose pull && docker-compose up is functionally identical. There's no good reason to add another flag for it.

[blockloop]

Imagine that git didn't have pull because git fetch && git merge origin/master is functionally identical.

[Amvoled]

Adding a pull: true tag could be useful for example if some of the images you use in your compose file are in your cache. docker-compose pull pull all of the images in your compose file, and that pull will fail if these images are in your cache but not in the repository.

[fulllove890]

+1

[JustinLivi]

One scenario in which docker-compose pull && docker-compose up becomes impractical is when you're using multiple docker-compose files. You can easily end up with a command like docker-compose -f docker-compose.test.yml pull && docker-compose -f docker-compose.test.yml up.

[xellsys]

We have a scenario in which we develop locally and we would only want to pull some of the images from remote. The one (or more) developed locally should remain untouched. In that case we are obligated to build/pull images manually before running docker-compose up.
A pull: true would be beneficial.

[lig]

@shin- what about rethinking your decision on this? I believe comments and reactions to them look self-descriptive enough.

[shin-]

No, I'm good. This is an open-source project, if you disagree with the conservative approach we take when adding features, you're welcome to fork it.

[alwye]

Agree on adding either a flag to the docker-compose up command or a parameter to the configuration. We use a base image with additional configuration that tends to change frequently during the development process. We want to create a fool-proof environment, where a developer can simply run docker-compose up without debugging what they don't need to debug.

I actually hit this thread after my colleague was checking my pull request and said it broke the build. And the reason being is that the base image was simply missing a couple of packages - but got executed in the final Dockerfile. It would be a good feature, but apparently not something you used, @shin- .

We'd be happy if you introduce this feature in the new version.

[kgalli]

@shin- I think @blockloop showed with the git example how convenient / useful the pull option is. Honestly I hope for such things it is not necessary to fork any project.

I totally understand the conservative approach but it feels like it is not even considered as an option anymore. Maybe it can be part of an upcoming version?

[pwFoo]

A pull: IfNotPresent would be nice. So it could be possible to use fallbacks like

1. use local image
2. pull if isn't local
3. build if not able to pull

[jrich523]

@shin- you keep asking for a reason why the && method wont cut it, and my reason is this. I use it for a "app" image for testing (Puppet PDK/Onceover). The compose file is part of the template repo so when a puppet dev (really ops folks) needs to make a new module they fork that repo. Jenkins runs the image for merge validation on that module repo (internally we have a jenkins plugin that handles the update for the jobs.) Now the folks who use this wont be docker experts, and having to tell them to do the && is an extra step that they can (and probably will) screw up. I dont see why it would be hard or what disadvantage it would create but this reasoning seems like a worthwhile reason to add it. It helps us developers send things out that require less directions and steps..

the short of it is.... to protect against laziness

[ostrolucky]

Here is better reason: && is synchronous. But docker-compose has built in great support for parallel executor which optimizes this stuff. docker-compose up --pull --build could start building image and run it as soon as it's pulled, instead of waiting for all images to be pulled and only then start building

[sgohl]

@shin is assuming one's using for ourselves on a very rare basis with rare updating Dockerfiles.
But if you update a Docker image every day that developers will use, it will rapidly lead to development problems if just one developer forgets to pull the image which he has to do every day (especially those who are not really familiar with docker, and that's actually not their business to know and remember it). Disaster programmed.
Is it a big problem to just add this option to the docker-compose.yml ?
I mean, it will not change other things, it just adds functionality. ..

it's the killer reason why I can't use docker-compose and need to write wrapper scripts with legacy docker run commands , but that's ugly.

[themanifold]

Trying to figure the circumstances under which this ticket was closed - would someone please elaborate? If it's any help, I am pro adding a --pull flag to docker-compose up.

[agnjunio]

So, here's a good use case:

I implemented a CI for my microservices project, which pulls new images to a registry when we develop new features in the backend service. The frontend team (which knows little about docker) needs to have a way to bring up the entire backend stack on their local machines, and they rely on the most updated images. If something breaks, only then they will remember to pull the images.

Now this is what happened: An entire development sprint went to fail because someone forgot to update the backend images, and developed an entire feature based on an old version. Blame on frontend team, but this could be avoided with this functionality (which I'll be doing using wrapper scripts).

[shin-]

@agnjunio That sounds really unfortunate, sorry. However, if the person forgot to run docker-compose pull, I'm not sure how them forgetting to use a hypothetical --pull flag is any less likely?

[agnjunio]

@shin- Sorry, I forgot to mention an important thing: The solution in my case is to have the tag pull: always inside the yaml, maybe inside image: options

[solsson]

@ET-CS from #3574 (comment):

I'd expect --build to bring everything to latest.

IMHO --build should pull; no need for any other flag or configuration. if you don't want it to pull then specify image version.

AFAIK that would block the use case with a build using a FROM that is the result of a depends_on build.

version: "3.4"
services:
  some-base-image:
    image: our/base-image
    build:
      context: ./base
  # This Dockerfile has FROM our/base-image
  coolthing:
    depends_on:
    - some-base-image
    build:
      context: ./coolthing

I'm in favor of a flag as suggested in #3574 (comment) and #3574 (comment)

[ET-CS]

@solsson
I am not sure I see why or what it's blocking in this use case.
please share more information about that if you can.

[ruanltbg]

@shin- The difference here would be that if I run docker-compose up --help I would receive a descriptive way in how to use the :latest image, instead nowadays I have to search in the doc or in the google that leads me to this thread and I need have to read all the comments to understand that docker-compose up does not do what I need/want, so now I need to run two command.

@agnjunio That sounds really unfortunate, sorry. However, if the person forgot to run docker-compose pull, I'm not sure how them forgetting to use a hypothetical --pull flag is any less likely?

Cordially

[ahannigan]

One of the main reasons we use docker-compose is the ability to place a docker-compose.yaml file in to a repository and have reproducible output when a developer pulls the repository and runs docker-compose up [service].

We use several services within our docker-compose files that perform tasks like running codegen and running a tool to dereference a JSON schema in to a single file.
Making sure these tools are up-to-date is critical especially if we update our codegen image to fix some common critical issue seen across all projects.

Having the ability to place:

services:
  codegen:
    image: myimage:latest
    pull: Always

would retain our ability to have a developer reliably run docker-compose up and get expected results, rather than having to supplement every single repository with documentation to remind developers to run a chain of commands or scripts to pull the latest availabe images before starting the apps.

It's not about "the functionality already exists to do this by running these commands", it's a better user experience.

Imagine when someone suggested adding --stop to docker-compose rm that the answer was "what's wrong with docker-compose stop myapp && docker-compose rm myapp, or if someone had requested the implementation of docker-compose down they were just asked why is docker-compose stop myapp && docker-compose rm -v myapp && docker-compose images -q myapp | xargs docker rmi ... impractical?

[jbauson]

This thread has been raised 2years ago and I still thought that adding a flag in the docker-compose.yml is the best way. In my case, we have like 37 services in a swarm, and updating 4-5 images from that total is not easy. I just created a shell for now that can monitor the change from the repository and do the pull for the specific image before recreating the service if it has been updated.

[jrynyt]

Another point here is that docker-compose up has a --quiet-pull option. When I was first trying to figure out if up included a pull, I assumed it did based on its presence. It goes to reason that not using --quiet-pull results in a verbose pull.

[eaglet3d]

Two years of people trying to convince the Docker Compose maintainers that having a --pull option would be a better experience without having to run a separate command. If the docker-compose maintainers just implemented the feature, to begin with, everyone's lives would be better. It seems to be clear that the current maintainers don't want to add this feature for the betterment of the community.

Maybe we should just fork docker-compose and update it ourselves.

[bean5]

If someone were to submit a PR, would it stands a chance if being accepted? This is open source. I've been close to looking into implementing it a few times myself. I assume it could be accepted, otherwise this role be closed, right?

[ktravelet]

I ran into this same issue and holy crap, a bunch of whiners in here. This is FREE open source software. Give the maintainers a break. I'm sure they have much more important things to work on than this. If anyone needs this so badly, why don't you open a PR? If the code is clean with minimal risk, I don't see why they wouldn't accept it.

[dvdgsng]

This issue should be re-opened since the discussion lacks a good reason to not implement this request. People are more likely to start working on open issues rather than closed ones.

[orodbhen]

The fact that this "closed" issue has remained so active does suggest that it has not been handled well.

Unfortunately, I've found that responses to issue posts on some GitHub repos are not very helpful. The tone is often terse, and suggests that feedback is less than welcome.

Some have pointed out here that this is an open-source project, and (at least most of us) aren't paying customers. However, it's also worth noting that submitting issue reports constitutes a significant time investment, even more so if you participate in the issue's resolution.

A user or developer, having spent time troubleshooting a problem, and found a workaround, will then decide if they want to spend additional time to report it. An unreceptive response from maintainers will likely result in them opting not to bother the next time around.

The software is not really "FREE", in the "free beer" sense. as we're all trying to participate in making it better. Having people willing to test your software and provide feedback is a valuable resource. Those with the requisite programming skills are even willing to contribute code, but they want some kind of indication that their contributions are welcome before spending time on it.

Obviously, comments that simply complain about a problem and demand that it be fixed aren't useful, but the majority aren't doing that, and comments like "it's open source, just fork it" are equally useless.

[jdalegonzalez]

@shin- Why was "--build" implemented? Is it different than docker-compose build && docker-compose up? Just trying to understand the philosophical different between --build (which got added) and --pull (which has been deemed redundant). Understanding the thought process might help me remember how things work. AND if the issue is opened, I'm happy to submit a PR. @EverybodyElse... is it really the "spirit of opensource" that if you don't like something you fork? I thought the spirit of opensource was "if you don't like something you a) help contribute to the requirements if that's where you are, b) help contribute to the code if you can" and that you only forked if your requirements were very clearly something only you would benefit from. ie I thought we benefit most when we share - but I'm happy to be educated here.

[alonsohki]

After two years of insisting, giving good reasons that are ignored and enormous community support towards having this actually implemented, I would say that this feature won't make it just because @shin- doesn't want to. No reason to keep insisting.

[SuperSandro2000]

There is one reason: docker won't refresh the images if one pull fails and there is no good reason not to do it.

[c9s]

I am looking for the kubernetes image pull policy in docker compose, would be great if the "pull" can be used.

[shin-]

Locking as this keeps getting derailed. We'll re-evaluate depending on the fate of docker/cli#1498

[shin-]

As an update, we've decided internally to look into adding a pull_policy parameter to service definitions. We still have to figure out what the options and exact syntax will be, but we hope it'll satisfy the needs expressed in this thread.