Reset properties inherited from parent image

[shykes]

When building an image I may want to reset some of its properties instead of inheriting them from the parent image. It makes sense to inherit all properties by default, but there should be a way to explicitly and selectively reset them when it makes sense.

This is a more generic solution to #2210 which only addresses expose.

[shykes]

Suggestions welcome for syntax.

[tianon]

The best I can come up with is corresponding commands like UNVOLUME or more generically -VOLUME (but that would add more confusion, and potentially even create the misconception that +VOLUME should work, and should work differently from just VOLUME).

I very definitely want such a thing (most especially for VOLUMEs). It's also a little disorienting that things like VOLUME apply to following RUN lines, but things like ENTRYPOINT don't. Sometimes that's very useful, and sometimes it's not, but a generic "disable previous X instruction" could solve the issues around that quite nicely.

[dergachev]

Is there a workaround for this in the meantime? I'm extending an image with an ENTRYPOINT (https://github.com/jagregory/pandoc-docker/blob/master/Dockerfile) and I need to unset the entrypoint. I tried using the following in my Dockerfile:

FROM jagregory/pandoc
ENTRYPOINT [] # this basically gets ignored (bug?)

FROM jagregory/pandoc
ENTRYPOINT [""] # this will make docker try to exec '' (the empty string)

FROM jagregory/pandoc
ENTRYPOINT ["/bin/sh", "-c"] 
# this will only work if docker run args are quoted:
#   docker run dergachev/pandoc "echo a b c"

Thanks!

[stormbeta]

I would definitely like to have some way of removing VOLUME points inherited from parent images.

For example, suppose I had a main image for an application that used an external mountpoint for persistent data, but I also wanted an image based on it that was pre-populated with test data instead. As-is, I can't do that if the parent image used VOLUME because any changes/additions to those directories, even if those changes are during a docker build, are lost on commit.

[tianon]

Just to update from @dergachev's comment, CMD [] and ENTRYPOINT [] were working last time I tested them recently, and should still be working (anything else would be ripe for bug filing).

[mmerickel]

You can reset all of the single-option commands via

ENTRYPOINT []
CMD []
USER 0
WORKDIR /

This should leave the remaining, un-resettable metadata as ENV, VOLUME, EXPOSE and maybe ONBUILD.

[codeon-nat]

(This is coming from #8709)

If I exposed sockets 9000-9002 in the parent, but needed to unexpose 9001 in the child, I'd then have to write in the style of "unsetting"

EXPOSE
EXPOSE 9000
EXPOSE 9002

which would work but

UNEXPOSE 9001

looks nicer.

An advantage being, that it doesn't affect any EXPOSEs from further up the inheritance chain, which I might want to add later.

[AshleyAitken]

+1 @codeon-nat

[icecrime]

This has been discussed in #8177, we're closing this for a lack of real world use cases.

[mitar]

Why is this being closed? There were 9 people commenting here. I think this would be a really useful thing to have. The real world use cases are being able to build upon existing images with ease. Sometimes you want to add property, sometimes you want to remove it. This is normal.

[kamermans]

I agree, for example, I'm extending the nginx image for an SSL offloader and I want to UNEXPOSE 80 but leave 443.

[seth-admittedly]

Being able to unexpose ports is pretty critical if you want to run multiple instances of, e.g., nginx. Without this feature, each image that inherits from nginx tries to expose 80 and 443, causing the error Bind for 0.0.0.0:80 failed: port is already allocated.

Nevermind, this was just poor configuration on my side.

[apatrida]

(April 15: "No real world use cases" I'm surprised you cannot imagine at least one and closed this)

I have a base image that exposes volumes or ports for optional software, then FROM that in another Dockerfile to make an image that should not expose anything it doesn't want to, or even things that it has uninstalled from the ancestor. Why would we NOT want to be able to remove these settings?

[download13]

I also have a use case for this. I want to be able to create an image containing a database snapshot, but all the mysql packages have VOLUME /var/lib/mysql set. It would nice to be able to turn off the volume the changes to the database made in my Dockerfile will stick with the image.

The only other option is to completely re-create a custom mysql image, but that somehow seems wasteful since plenty of other people have already put together better default mysql servers than I could.

[tomheinan]

Adding an additional use case - I'm inheriting from the official RabbitMQ image, but I only want to expose websocket ports (80 and 443) and not the default AMQP port (5672). Seems like this should be a pretty reasonable thing to want to do?

[praqma-thi]

Adding another use case. I want to build a git test environment using the gogs image but it's tedious to have the data persist since it's all stored in a volume. It'd be great if I'd be able to simply UNVOLUME the volume and build my image after setting up the environment.

[krasi-georgiev]

+1

inheriting from the official php and want to use sockets instead of ports so need to remove the exposed 9000 port

[macropin]

Anyone who has used Docker in a non-trivial capacity will have found these limitations with inherited containers.

@shykes @icecrime how is this now closed? Is it too hard to solve with the current syntax and need for backwards compatibility? Whats the plan?

[ianmiell]

+1 - real world use case for EXPOSE override here.

[alexeyp0708]

Not only is this potentially insecure but also a hacky workaround at best. I don't think anyone is going to be doing this for a production system, I'd rather stick to the "keep the Dockerfile in sync" method instead.

I didn’t offer it for production.
Anyone in the know knows what they are doing. This for workspace

[alexeyp0708]

But you can still wait 10 years

[alexeyp0708]

I think it’s worth highlighting simple rules:
If you use a third-party image, then you must accept its rules as they are. This is necessary for compatibility and reducing technical debt.
If you are trying to change something in the image by inheritance, then it is better to create a new image from scratch.
Images for development stack - create whatever your heart desires.

[alexeyp0708]

Many people don’t understand why "volume" is in the Dockerfile.
For example, I deployed MYSql image from command
run -d --name mysql --network db_net --ip 172.19.32.2 mysql:8.3
The image mounts the volume /var/lib/mysql

Hundreds of entries have been made to the database through another image. The container stopped and then moved away.
But the data remains.
I think the meaning is clear!
People don't always create new images or mount their own volumes.
And how much code will depend on such default actions, only Code God knows.
Removing VOLUME is fraught with the following consequences:

• change scripts
• change image assembly
• change the behavior of orchestrators.

I understand the developers. They follow design patterns where a child object can extend the image, but is not allowed to change it. (open-closed principle in SOLID). (Although I think that not everything is ideal when inheriting)

Therefore, additional Directives in DOCKERFILE that will allow you to break the behavior of the parent image are contraindicated.

Imagine that you need to change one container to another and maintain compatibility.
However, knowledge of how containers are used is beyond your understanding.
FOR example, before you used the mysql:8.3 image, and now you use the child image my_mysql, and in this container you break backward compatibility by deleting the mounted volume of the parent image.

[felipecrs]

Another good reason why VOLUME in dockerfiles exist is the official docker:dind. In there the /var/lib/docker needs to be a volume to work correctly, and with VOLUME, users don't need to pass --volume themselves.

[lucasbasquerotto]

Many people don’t understand why "volume" is in the Dockerfile.

Except for some specific use cases like docker:dind and speeding up build time in multi-stage builds (to not store/change files in the Copy-on-Write layer), there's no good reason other than metadata for volumes in dockerfiles.

If you are running an image relying on its anonymous volumes, you are doing it wrong.

The reason official images still have it is mainly backwards compatibility.

Some images removed it, like oracle: oracle/docker-images#640 (comment)

Other images kept it, like MySQL: docker-library/mysql#255 (comment)

The reason mysql keeps the VOLUME directive is basically what the comment in the link above says:

I feel like removing the volume would break many users that rely on the volume when using docker-compose.

It's just that. And I think it's fine. But it's not about volumes in the dockerfile being good, or because the image would not work if UNVOLUME was used in a Dockerfile that extends it. If you want persistence, the image may really not work with UNVOLUME, but you should specify a volume when running the container anyway. Would you use a database in production not knowing where the data is stored?

That said, an option to completely disable image volumes (allowing only volumes defined in runtime) when running a container would be much better than UNVOLUME. You would not need to cherry-pick each path that needs UNVOLUME in each image, and you also would not need to create a Dockerfile to extend the image at all. Just disable the behaviour. I don't know if this is something easy to do on the docker side tough, but it would be the best solution.

[Gunni]

Why is nobody talking about removing EXPOSE statements for example if I take the nginx container and want to remove port 80 listening port and put it on Port 8080.

[felipecrs]

If you are running an image relying on its anonymous volumes, you are doing it wrong.

I disagree. As I mentioned above there is clearly a use case for it. And even if docker:dind is the only valid use case in the world (I agree MySQL should NOT have anonymous volumes declared), it does not mean it's done wrong.

An anonymous (or not) volume is necessary for some workloads, and in such case anonymous VOLUMEs helps avoiding handling volumes manually on each container execution for people that do not actually care about the underlying data within such volumes. Which is SUPER useful and a time saver for docker:dind.

[lucasbasquerotto]

I disagree. As I mentioned above there is clearly a use case for it. And even if docker:dind is the only valid use case in the world (I agree MySQL should NOT have anonymous volumes declared), it does not mean it's done wrong.

The thing about dind and some other use cases I mentioned in the beginning of my comment, and said they can be considered exceptions. That statement you quoted I meant for basically every other use case (which should be almost all cases btw). Perhaps I should have been more clear about it.

That said, I'm not advocating for removing VOLUME and EXPOSE for now, because I don't think that would be feasible, at least in the short and medium term, and would break things for people who rely on them.

My only hope is for the docker team to add an option to disable them if you want it. Like I said previously, it could be with something like docker run --no-image-volumes --no-image-ports ... (and correponding options for docker compose). These options would ignore VOLUME and EXPOSE defined for an image, as if they were never defined, when running a container based on the said image.

[simaotwx]

Another good reason why VOLUME in dockerfiles exist is the official docker:dind. In there the /var/lib/docker needs to be a volume to work correctly, and with VOLUME, users don't need to pass --volume themselves.

However, this can still exist even if something like UNVOLUME exists. The real trouble is for downstream users who wish to specify their own volume and not rely on VOLUME. I totally dislike that volumes are by default located at a managed location like /var/lib/docker/volumes making it a PITA to manage, especially when you want to move those volumes to another system and now you need to start digging around and finding out which volume belongs to what container and what can be moved and what can be deleted.

[simaotwx]

It's just that. And I think it's fine. But it's not about volumes in the dockerfile being good, or because the image would not work if UNVOLUME was used in a Dockerfile that extends it. If you want persistence, the image may really not work with UNVOLUME, but you should specify a volume when running the container anyway. Would you use a database in production not knowing where the data is stored?

That said, an option to completely disable image volumes (allowing only volumes defined in runtime) when running a container would be much better than UNVOLUME. You would not need to cherry-pick each path that needs UNVOLUME in each image, and you also would not need to create a Dockerfile to extend the image at all. Just disable the behaviour. I don't know if this is something easy to do on the docker side tough, but it would be the best solution.

I 100% agree with this and it's the exact reason why I'm voting for something like this to be implemented.
Everyone should know where their data is stored. I've destroyed a local development environment after setting up a new SSD and moving the data, only to find out that those managed docker volumes are stored on many, many different, hard to follow btrfs subvolumes, completely cluttering up my system and now I have to set up everything again.

So, from then on I am specifying volumes explicitly, always, no matter what.

[simaotwx]

If you are running an image relying on its anonymous volumes, you are doing it wrong.

I disagree. As I mentioned above there is clearly a use case for it. And even if docker:dind is the only valid use case in the world (I agree MySQL should NOT have anonymous volumes declared), it does not mean it's done wrong.

An anonymous (or not) volume is necessary for some workloads, and in such case anonymous VOLUMEs helps avoiding handling volumes manually on each container execution for people that do not actually care about the underlying data within such volumes. Which is SUPER useful and a time saver for docker:dind.

However, this could end up cluttering up your system hard drive, and you could be unaware of it, wondering where all those gigabytes went, just to find out that some image you ran at some point had unnecessarily launched gigabytes of data onto your system.

[alexeyp0708]

I came up with a lifehack But special conditions are needed docker run --privileged I think you'll understand with an example Dockerfile

...
...
RUN apt-get -y install util-linux

docker run --privileged --mount type=tmpfs,dst=/volume/path --entrypoint /bin/sh -it myImage:test -c "umount /volume/path;bash"

I’ll try another option via run --device , if it works I’ll post

I think this is the only way to disable the volume (via ENTRYPOINT) of the inherited image in order to work with the native directory of the image (even at least at the stage of building your own image).
UPDATED: This is necessary when you really need to detach the volume.
Some behavior has been fixed. Now VOLUME is metadata and will be mounted before the container is running.
#33842 (comment)

In the best case, it is make a stub for the volume if the directory data is not used -run --mount type=tmpfs,dst=/volume/path

Regarding --privileged.
You have 3 options to grant privileges.

1. option run --privileged - all privileges
2. option run --cap-add=SYS_ADMIN - privileges for administration only
3. option is to create your own profile.
   https://docs.docker.com/engine/security/apparmor/

Everything else is demagoguery.

[alexeyp0708]

@Gunni

Why is nobody talking about removing EXPOSE statements for example if I take the nginx container and want to remove port 80 listening port and put it on Port 8080.

The doc states https://docs.docker.com/reference/dockerfile/#expose

The EXPOSE instruction doesn't actually publish the port. It functions as a type of documentation between the person who builds the image and the person who runs the container, about which ports are intended to be published. To publish the port when running the container, use the -p flag on docker run to publish and map one or more ports, or the -P flag to publish all exposed ports and map them to high-order ports.

https://docs.docker.com/engine/reference/run/#exposed-ports

The -P flag only publishes port numbers that are explicitly flagged as exposed, either using the Dockerfile EXPOSE instruction or the --expose flag for the docker run command.

https://docs.docker.com/compose/compose-file/compose-file-v2/#expose

https://docs.docker.com/compose/compose-file/compose-file-v2/#ports

[alexeyp0708]

And as an option, a way to rebuild the image.
#3465 (comment)

[alexeyp0708]

However, this could end up cluttering up your system hard drive, and you could be unaware of it, wondering where all those gigabytes went, just to find out that some image you ran at some point had unnecessarily launched gigabytes of data onto your system.

Do run --mount type=tmpfs,dst=/volume/path
Оr docker volume prune

[felipecrs]

However, this could end up cluttering up your system hard drive, and you could be unaware of it, wondering where all those gigabytes went, just to find out that some image you ran at some point had unnecessarily launched gigabytes of data onto your system.

Do run --mount type=tmpfs,dst=/volume/path Оr docker volume prune

I'm much rather this as opposed to having to specify --volumes myself. But surely both cases are valid.

[alexeyp0708]

I'm much rather this as opposed to having to specify --volumes myself. But surely both cases are valid.

In any case, such decisions will have to be thought through.

If you carefully read the post #8177 (comment), you will understand that there will be no solutions.

[felipecrs]

Oh God. That's depressing. No point to continue the discussion then.

[alexeyp0708]

@thaJeztah
A new topic has been opened in moby/buildkit
moby/buildkit#4802
Based on the comment #8177 (comment)
Please close and lock the topic.

[q0rban]

Please close and lock the topic.

Don't we still want this open for unsetting these values via docker run / docker container create?

[alexeyp0708]

#8177 (comment)

see admin's response #8177 (comment)
This issue is best discussed in mody/buildkit since in the legacy Dockerfile builder "moby/moby" the changes will only affect bugs.
Now I propose to take a consolidated approach to the problem in a new branch,
so that the buildkit developers pay attention to this

[alexeyp0708]

Please close and lock the topic.

Don't we still want this open for unsetting these values via docker run / docker container create?

Perhaps this will affect on docker run, but I think only when there is a solution in Buildkit.
But for this you can then create a new topic.
In any case, we need to unite in a new topic to affect changes in "moby/buildkit"

[lucasbasquerotto]

@alexeyp0708 I think it's best to discuss here actually, or in both places, at least for volumes, because I think removing volumes defined in images should be able to be done in docker run. This should not be restricted to buildkit or even Dockerfiles.

Even if an image was created with docker container commit (without a Dockerfile), I should be able to remove image volumes in docker run.

From what I see, image volumes create a mount somewhere when a container is created from the image. I think the best way to solve it is to just add an option in docker run to not create the mount in this case.

That said, creating a Dockerfile inheriting from a parent image with some instruction like UNVOLUME would solve it too, but a docker run approach would avoid the need to extend the original image, and could skip the build step entirely.

[alexeyp0708]

@lucasbasquerotto

I think it's best to discuss here actually, or in both places, at least for volumes, because I think removing volumes defined in images should be able to be done in docker run. This should not be restricted to buildkit or even Dockerfiles.

Perhaps I was hasty in jumping to the conclusion was to close the topic.
Today the bulk of the participants are in this thread.

Studying discussions the topic, I came to the conclusion that there were mainly proposals to improve the Dockerfile.

I did not see that the participants were able to professionally reproduce their difficulties so that the developers would pay due attention to this.

I also got the impression that the developers stopped paying attention to this.
At least I didn't see any questions from the developers.
Now this is a discussion platform where there are no constructive proposals and solutions.
The topic is already 10 years old.
New topics with constructive and detailed idea will interest developers much faster than a discussion of 500 or more messages. The very fact of studying such a long topic discourages interest in diving into details.
In most cases, this topic contains only wishes and meaningless discussions.

[alexeyp0708]

@lucasbasquerotto

Even if an image was created with docker container commit (without a Dockerfile), I should be able to remove image volumes in docker run.

I agree with you that we need a command that can disable the parent volume.I suggested this idea a little higher. run --mount type=none, dst=/dir/volume
#3465 (comment)
There were several other ideas for unmounting volumes VIA docker run.
But I think if they enter mount type=none it will be a better idea.
none indicates that the folder is protected from any volume mounting.
Arguments:

• There is a new type none in the new command -- mount type = (volume|bind|tmpfs|none), which will be easier to implement. All them need to do in code is write a “dummy” driver that does nothing. But by specifying such a driver, the system will automatically override the default volume connections to this a driver.
• when a new type =none appears, it can be legally specified for docker compose in compose.yml

volumes:
      - type: none
        target: /data

[alexeyp0708]

I think even a simple workaround is to write a plugin that implements a stub driver(simulates connection to volume).
https://docs.docker.com/engine/extend/plugins_volume/
https://docs.docker.com/engine/extend/plugin_api/

(updated)
I read a little about the plugin documentation.
Unfortunately, this path is unlikely to be implemented.
Plugins work over sockets and cannot affect the container in any way. This means that the volume will definitely be mounted inside the container.