Any php program in a linked container can read the mysql root password

[jlj]

Note: this is a copy of a message about this issue that I wrote as a comment to docker issue #5169 (closed) see moby/moby#5169 (comment)

Although the docker documentation now clearly includes a warning about env variables being propagated to linked containers, this still looks like a real security problem to me, because many images, including official images pass secret information though env variables, and there still doesn't seem to be a good way to pass a private variable to a container at runtime.

This can be illustrated by a very simple php / mysql application using a database container,using the official mariadb image, linked with a php container based on the official php image.

The result is simply that any php program has access to the mysql root password via the env variables:

And really, this is not desirable!

Ok, this could be considered as a bug of the mariadb image and I will also enter it as such...
But my real feeling is that it looks more like a general docker issue.

[yosifkit]

This is how we can just --link an sql database to a wordpress container and it just works.

One possible solution is to only use the env for setup and then use a second container to --link to the database and setup the real users, changing the root password. That would make the envs not matter after initial setup, but then you have to configure all of your connecting containers with their users and passwords.

[tianon]

You could also use "initdb" (or just a normal connection after initial setup) to change the root password. We have to have _something_ to initialize the password to, and it can't really be interactive, so we don't have a lot of options here.

[jlj]

I agree that you don't have many options here for setting the mysql root password.

However, having the db root password visible in clear in the linked containers is a major security issue and makes completely meaningless the creation of other restricted users (like the one associated to the optional default database in the image).

Therefore, even if it is far from perfect, a solution for this could be to generate a random password when MYSQL_ROOT_PASSWORD is not specified on the command line, and to echo it in the container's log.

This means replacing in docker file-entrypoint.sh

    if [ ! -d "$DATADIR/mysql" ]; then
        if [ -z "$MYSQL_ROOT_PASSWORD" -a -z "$MYSQL_ALLOW_EMPTY_PASSWORD" ]; then
            echo >&2 'error: database is uninitialized and MYSQL_ROOT_PASSWORD not set'
            echo >&2 '  Did you forget to add -e MYSQL_ROOT_PASSWORD=... ?'
            exit 1
        fi

by

    if [ ! -d "$DATADIR/mysql" ]; then
        if [ -z "$MYSQL_ROOT_PASSWORD" -a -z "$MYSQL_ALLOW_EMPTY_PASSWORD" ]; then
            MYSQL_ROOT_PASSWORD=$(pwgen -s 12 1)

            echo "========================================================================"
            echo "  Generated MYSQL_ROOT_PASSWORD for this Docker container: $MYSQL_ROOT_PASSWORD"
            echo "========================================================================"
        fi

This has the advantage of keeping the compatibility with the existing command line, and running mariadb as docker run --name database -d mariadb has the expected result, i.e. that the root password is not visible anymore as an environment variable in a linked php-based container.

If you are inline with this proposal, I can create a pull request; otherwise I'll just use it for my own needs... :-)

[skyred]

I just realized this security problem today. This problem is that the official docker mariadb library ships with a pretty big security hole as default. I cannot agree with #21 (comment) that conviennce takes higher priority than security in this case.

I am proposing that we simply make MYSQL_ROOT_PASSWORD optional. This would make the two use cases work:

1. When people want " just --link an sql database to a wordpress container", they can use MYSQL_ROOT_PASSWORD env variable.
2. When people want take full advantage of MySQL permissions & roles, they need to get in mariadb instance after docker run to initialize their own root password. We can use the code from Any php program in a linked container can read the mysql root password #21 (comment)

[jlj]

Please be aware that this root password leak issue is not specific to the mariadb docker image. For example I checked the mysql official image and it presents the exact same issue (which is not surprising, since both mariadb and mysql have almost identical Dockerfiles).

[thaJeztah]

I opened up an issue in docker a while ago to design/write up alternatives for managing "secrets"; see moby/moby#13490. The current behavior is "by design", and is useful for "automatic" configuration of linked containers, but I fully agree that leaking these env-vars to linked containers is far from ideal.

[skyred]

@thaJeztah please enlighten me what's the downside of making MYSQL_ROOT_PASSWORD optional? The "automatic" configuration can still work as before, right?

[thaJeztah]

@skyred sorry, just wanted to add a reference to general optimizations that could be made, w.r.t;

Although the docker documentation now clearly includes a warning about env variables being propagated to linked containers, this still looks like a real security problem to me

I agree on making the MYSQL_ROOT_PASSWORD optional.

[iBobik]

Make MYSQL_ROOT_PASSWORD optional it good solution.

Don't forgot to document it in README, to make everybody understand that they should use MYSQL_ROOT_PASSWORD only if they know what they do.

[yosifkit]

MYSQL_ROOT_PASSWORD is optional, just set MYSQL_ALLOW_EMPTY_PASSWORD to something non-empty. I'll be updating the docs for the mariadb image to reflect this; MYSQL_ALLOW_EMPTY_PASSWORD has been available since #4 😳.

[iBobik]

Nice, going to use it.

But why two variables for use case, where one is enough and more intuitive?

Jan Pobořil

2015-09-23 18:50 GMT-05:00 yosifkit notifications@github.com:

MYSQL_ROOT_PASSWORD is optional, just set MYSQL_ALLOW_EMPTY_PASSWORD to
something non-empty. I'll be updating the docs for the mariadb image to
reflect this; MYSQL_ALLOW_EMPTY_PASSWORD has been available since #4
#4 [image: 😳].

—
Reply to this email directly or view it on GitHub
#21 (comment)
.

[jlj]

@yosifkit Sorry, but MYSQL_ALLOW_EMPTY_PASSWORD can hardly be proposed a solution for the security problem here: replacing make sql root password public for every linked container by do not set a sql root password does not seem like a real improvement to me.

What would rather be needed is a way to set the sql root password in such a way that it is not accessible to other linked containers.

[md5]

@jlj This repo isn't really the place to debate the need for such a feature. Looks like you've already commented on moby/moby#5169, which is the most recent discussion I've found specifically dealing with that topic. As far as I know the "links v2" proposal mentioned in that issue never happened. This comment mentions that a replacement proposal was going to be made public at some point, but I don't recall ever seeing one. Others may know better. This issue is also a good overview of the past proposals and possible future roadmap for handling secrets in Docker: moby/moby#13490

[md5]

@jlj I also see now that you're already aware of moby/moby#13490

[jlj]

@md5 The point of this issue is not to debate about docker features. Actually, the fact that, if using this official mariadb image, any php program running in a separate-but-linked container has a direct clear access to the SQL root password looks like a serious security concern to me.

Additionally, and this may seem naive, I thought that official docker image were supposed to be representative of the best practices, e.g. the ones listed in the post you mentioned moby/moby#13490. And if the people in charge of this repo consider that the mysql root password shall not be considered as a secret ,and thus can be put in an environment variable without any inconvenience, this is not a problem for me (I just won't use this image), but it seems reasonable that other users are aware of this when selecting docker images for building their system. :-(

[md5]

@jlj That's what I get for commenting on GitHub issues past my bedtime and not fully reading the thread first. Sorry about that.

I guess what I should have said is that you're not really offering a concrete proposal. Given the state of Docker secret management in moby/moby#13490, I only see a few ways of dealing with this issue using the current features:

1. Change the entrypoint to take the password on the command line somehow. This would involve having the entrypoint interpret some arguments itself as opposed to the current situation where it only passes arguments through to the underlying CMD.
2. Put the password into a file or directory mounted with --volume (or possibly a Hashicorp Vault or Keywhiz filesystem mount) and tell MySQL to read it from there.
3. Tell users to set the root password after the container is started. A related approach would be to set the root password in an entrypoint initdb.d script, but you'd have to ensure that it runs last to avoid breaking other entrypoint initdb.d scripts.
4. Use a preinitialized data directory.
5. Avoid using --link.

None of these seem like great solutions to me, though the first one seems slightly better in some ways. The use of a key management server in the second bullet seems good as well, but difficult to recommend for casual users.

Since you've said that your solution will be to avoid using this image, how do you intend to set the root password without exposing it to linked containers? My intent in linking to moby/moby#13490 was to point out that the Docker team has not yet given any useful guidance on how to avoid this scenario and there aren't really any great alternatives given the current feature set.

[jlj]

@md5 The concrete proposal I've made is to generate an random password when MYSQL_ROOT_PASSWORD is not specified on the command line (see #21 (comment) 20 days ago). This is a 3-lines change in the existing CMD script, it is fully compatible with the current command line parameters, and this is what I am actually using in my projects.
I agree it is not perfect, but it does the job.

[md5]

@jlj Thanks! I should have known that! I'll read the thread more closely next time before commenting. 👍

[thaJeztah]

Actually, the new networking/linking is currently available as part of the "experimental" builds of Docker. See https://github.com/docker/docker/blob/master/experimental/networking.md for more info

[md5]

Thanks @thaJeztah. Looks like moby/moby#14083 is the place to track that feature?

[iBobik]

Is not enought to hide just root password? So it could be randomly generated and if admin needs to use root, he can open shell into container and use mysql client under root.

Jan Pobořil
http://honza.poboril.cz/

24 Sep 2015 v 08:24, Mike Dillon notifications@github.com:

@jlj That's what I get for commenting on GitHub issues past my bedtime and not fully reading the thread first. Sorry about that.

I guess what I should have said is that you're not really offering a concrete proposal. Given the state of Docker secret management in moby/moby#13490, I only see a few ways of dealing with this issue using the current features:

Change the entrypoint to take the password on the command line somehow. This would involve having the entrypoint interpret some arguments itself as opposed to the current situation where it only passes arguments through to the underlying CMD.
Tell users to set the root password after the container is started. A related approach would be to set the root password in an entrypoint initdb.d script, but you'd have to ensure that it runs last to avoid breaking other entrypoint initdb.d scripts.
Use a preinitialized data directory.
Avoid using --link.
None of these seem like great solutions to me, though the first one seems slightly better in some ways.

Since you've said that your solution will be to avoid using this image, how do you intend to set the root password without exposing it to linked containers. My intent in linking to moby/moby#13490 was to point out that the Docker team has not yet given any useful guidance on how to avoid this scenario and there aren't really any great alternatives given the current feature set.

—
Reply to this email directly or view it on GitHub.

[md5]

@iBobik That's not possible if the user specifies it with -e MYSQL_ROOT_PASSWORD and uses --link. Docker automatically makes all such environment variables (and all those specified in the image with ENV) available to linked containers.

[tianon]

It is really trivial to change the root password after the fact, though. Perhaps a simple example of how to do that would be a good addition to the docs.

[thaJeztah]

@md5 correct; I think there are some related issues as well. Be aware that it's still "experimental", which means that the design can/will still change, based on feedback.

[mgmax]

While working on another project I figured out a backwards-compatible solution, for which you don't need to change your environment files. It uses a volume-mount to bring the environment file inside the container. An entrypoint script reads the environment file from $ENVFILE and starts the command:
fau-fablab/app-server@6fa1a6a

This solution is backwards compatible - if you don't set the ENVFILE variable, it will just do nothing. All existing env variables are preserved.

Copy-and-pasting this code to your entrypoint script should work. Then you just need to change your calls from
docker run ... --envfile=/my/env/path ...
to
docker run ... -e ENVFILE=/env -v /my/env/path:/env:ro ...

[iBobik]

How about to change behavior of variable MYSQL_ROOT_PASSWORD for empty value? Currently it sets empty password, so everybody can login. More secure behavior will be to disable login (if it is possible) or set password to random string (and not expose it outside container).

[thaJeztah]

Note that the new networking options in docker 1.10 don't require to setup links between containers (as long as they're in the same network), and environment-variables are not created/shared between containers. Also see this section in the documentation; https://docs.docker.com/engine/userguide/networking/work-with-networks/#linking-containers-in-user-defined-networks

[asokani]

You can move database files to your host environment like this:
sudo docker run --name some-mariadb -v /var/mariadb:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mariadb
then stop and rm this instance and run mariadb again without the root password parameter, because db files are already initialized:
sudo docker run --name some-mariadb -v /var/mariadb:/var/lib/mysql -d mariadb

[fabiomontefuscolo]

Well, I'm hosting a PHP app that is not mine. I run this app overwriting this MYSQL_ROOT_PASSWORD.

docker run --name blank --link mysql \
                 --restart=always  \
                 -e MYSQL_ENV_MYSQL_ROOT_PASSWORD=xxx \
                 -v /srv/ftp/blank/:/var/www/html/ \
                 -p 8080:80 montefuscolo/php:7

But I have to remember to hide password on every docker I link with mysql.

[thaJeztah]

@fabiomontefuscolo create a custom network (docker network create mynet), and when starting containers, use that network;

docker run --name blank --link mysql \
                 --net=mynet \
                 --restart=always  \
                 -e MYSQL_ENV_MYSQL_ROOT_PASSWORD=xxx \
                 -v /srv/ftp/blank/:/var/www/html/ \
                 -p 8080:80 montefuscolo/php:7

environment vars are not shared in custom networks

[COLABORATI]

Is this real? Is this kind of a parody or a joke?
If it is actually real, then please put this on top of the "official mariadb docker image" description, thanks!
https://hub.docker.com/_/mariadb/

[alizowghi]

@thaJeztah When I use your solution, in new container I haven't any ip record for container in /etc/hosts:

with default network:

127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.2 mysql fa259323f008
172.17.0.7 29414bab4b67

and with my network:

127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.2 fa259323f008 #this is mysql container

[fabiomontefuscolo]

@alizowghi I saw in your reply a line like this:

172.17.0.2 mysql fa259323f008

This is generated by --link mysql parameter. You don't need to overwrite variables if you use the @thaJeztah answer above, because when you use custom networks, env vars will not be shared.

[alizowghi]

@fabiomontefuscolo my problem is lack of this line "172.17.0.7 29414bab4b67" in custom network. in this case my services in container can't run well.

[fabiomontefuscolo]

@alizowghi I don't know how to solve this, because it just works for me. Maybe, as a second try you can configure a dns server container, like jderusse/dns-gen, but I use it only on my development environment.

After the trick I learnt with @thaJeztah, I read more about docker-compose.yml and I could deploy something like this using custom networks. And things just work, my /etc/hosts is ok.

version: '2'

services:
  proxy:
    image: jwilder/nginx-proxy
    container_name: nginx-proxy
    restart: always
    ports:
      - "80:80"
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
    networks:
      - front

  mapas_bh:
    image: hacklab/mapasculturais
    restart: always
    environment:
      - PG_HOST=postgis
      - VIRTUAL_HOST=example.com.br
    networks:
      - front
      - back

  postgis:
    image: mdillon/postgis
    container_name: postgis
    restart: always
    networks:
      - back

networks:
  front:
  back:

[thaJeztah]

@alizowghi in user-defined networks, Docker now uses DNS based resolution of the linked containers, and containers on the same network. This is all handled by an embedded DNS server; https://docs.docker.com/engine/userguide/networking/configure-dns/#/embedded-dns-server-in-user-defined-networks

[alizowghi]

@fabiomontefuscolo , @thaJeztah thank you for your responses, I got it now, my problem was employment of two containers in two different networks.

@thaJeztah it's a very nice solution, thank you very much

[yosifkit]

Closing old issue. This is really an issue with --link in Docker and not the MariaDB images. With #89 you can use files with Docker secrets so this issue is irrelevant on production deployments. You can also just use Docker networks, instead of --link to avoid the issue.