Incorrect mode for docker-compose secret file

[lonix1]

According to the docs, the permissions for secrets files:

The default in Docker 1.13.1 is 0000, but is be 0444 in newer versions

And according to a previous issue, this was fixed.

But this is not the case for me.

• I have a secret file on the host with permissions 400. It is referenced in docker-compose.yml for a mariadb service. But mariadb errors with "permission denied".
• If I change the host secret file to 444, then it works, but of course that's a bad idea.

I thought 444 (for the container secret file: /run/secrets/foo) was supposed to be the default mode set by docker?

ubuntu 19.04
docker --version = Docker version 19.03.2, build 6a30dfc
docker-compose --version = docker-compose version 1.24.1, build 4667896

[thaJeztah]

Docker Compose doesn't support real (swarmkit) secrets, and imitates them by bind-mounting the file directly into the container (which means that permissions on the host are the same as in the container).

You can change the ownership of the file on the host to match the uid/gid of the user in the container, but otherwise I don't think there's much that can be done unfortunately

I'll close this issue because of the above, but feel free to continue the conversation

[lonix1]

Thank you for clarifying.

The problem is the "docker compose" and "docker swarm" products use the same file - docker-compose.yml - so for someone who only needs a single-server deployment, it becomes confusing. I didn't realise what I was reading in the docs (secrets) does not apply to me.

[lonix1]

@thaJeztah Follow-up question if I may.

I read it's easy to upgrade from "docker compose" to "docker swarm", so I thought I'd try it as a single-server stack. I added replicas: 1 to the compose file.

So the only change is
docker-compose up
to
docker swarm init; docker stack deploy -c docker-compose.yml foo

But it complains about the secret file on the host:

open /foo/bar/secretfile: permission denied

The secrets file (on the host) is owned by root:root and has 0400 permissions. Is this correct?

[thaJeztah]

The secrets file (on the host) is owned by root:root and has 0400 permissions. Is this correct?

Yes, that's likely expected; what happens is that the docker CLI attempts to open the file to read its content (which fails if the current user / user that runs the docker stack deploy doesn't have access).

The cli (if succeeded to read) will then send the content of the secret file to the daemon, and store it as a secret.

As an alternative, you can create the secret up-front (docker secret create ....) as a user that does have read-access on the file (or create it from stdin), and then, in the compose-file reference the secret that was created, instead of referring to the secret file in the compose file

[lonix1]

@thaJeztah I assumed I could use root:root and 400 because the user is a member of the docker group, and the daemon runs as root. I guess that was wrong.

I came up with a workaround. The secrets file on the host is:

$ sudo chown root:docker my_secrets_file
$ sudo chmod 440 my_secrets_file

This is a compromise - the secrets file is not locked-down as before, however it is accessible only to root and sudo-less docker users.

Do you believe this is a reasonable approach / is this how it's typically done?

[thaJeztah]

I think ideally the file would not be stored at all (as plain text), and used in a configuration management system or a password manager, then the secret created in advance.

Docker doesn't need access to the file once it's been stored as a secret (docker secret create)

If you do need the file locally, you can store it with 0400 permissions somewhere in your own home directory, so that only you (the person running docker stack deploy) has access to it.

[lonix1]

Thanks for your advice!

[niloct]

Bumping this thread.

What if an attacker gain access to reading "/run/secrets/my_secret" in anyway in the container ?

Even if I run the microservice under a non-root user, the 444 permissions on the secret enables anyone who gained access to the directory to reading the secret.

[thaJeztah]

@niloct if a container is compromised, that's not any different than a non-containerised situation (e.g. a VM or host compromised); take similar actions in that case (rotate secrets, consider data that was accessible with the secret to be compromised).

[niloct]

Hi, thanks for replying :)

Considering that in this scenario the container doesn't run under root, and the user hasn't sudo privileges, even if he compromises the container he wouldn't be able to see the secret.

Right ?

[hungpham2411]

Hi, thanks for replying :)

Considering that in this scenario the container doesn't run under root, and the user hasn't sudo privileges, even if he compromises the container he wouldn't be able to see the secret.

Right ?

This is exactly what I'm stuck in.
I deploy the ELK stack with ssl credentials managed by Docker secrets in Swarm and I don't use Dockerfile to custom ELK services because I don't want another container repository for the custom build.
However, I think this is the problem with original Elasticsearch docker image. They should add user elasticsearch to gid 0.

[thaJeztah]

However, I think this is the problem with original Elasticsearch docker image. They should add user elasticsearch to gid 0.

This ticket was about docker-compose, which does not use swarm services.

If you're using swarm services, you can set the uid, gid and permissions when attaching a secret to service.

Looking at the elasticsearch Dockerfile (https://github.com/elastic/dockerfiles/blob/v7.10.1/elasticsearch/Dockerfile#L70-L73), the elasticsearch user has uid:gid 1000: 1000, so if you would mount the secret with uid=0 (root) and gid=1000 (elasticsearch group), then permissions 0440, it should have access.

Create a secret

echo "foobar" | docker secret create mysecret -

Create a service using the secret, that mounts it at /run/secrets/foo, with owner 0:1000, and 0440 permissions;

docker service create --name myservice \
    --secret source=mysecret,target=foo,uid=0,gid=1000,mode=0440 \
    nginx:alpine

exec into a container of the service as user 1000:1000, verify ownership/permissions are correct, and that user 1000 is able to read the secret;

docker exec -it --user 1000:1000 myservice.1.of0ct8765bouq1ystjd7ya0fi sh

$ ls -la /run/secrets/foo
-r--r-----    1 root     1000             7 Mar  4 12:10 /run/secrets/foo
$ cat /run/secrets/foo
foobar

[hungpham2411]

If you're using swarm services, you can set the uid, gid and permissions when attaching a secret to service.

Thanks for the detailed answer. You're so kind.
Before coming here. After the whole day of digging and searching, I'm pretty sure the problem wasn't setting uid:gid or directory /run/secrets itself. They have the right access mode. There is something wrong with Elasticsearch security, I think. No matter what I try, I always get an access denied error if xpack.security.enabled is true in Elasticsearch config file:

java.security.AccessControlException: access denied ("java.io.FilePermission" "/run/secrets" "read")

This ticket was about docker-compose, which does not use swarm services.

Since you noticed, I'm stopping here. Thank you again!

[thaJeztah]

Hmm.. not sure what's causing that; looking at permissions, they look ok; I'm able to list the contents of the /run/secrets directory (running as user 1000:1000)

docker exec -it --user 1000:1000 myservice.1.zkgx21wteg05yxsd0zf4t4llz sh
/ $ ls -la /run/secrets
total 12
drwxr-xr-x    2 root     root          4096 Mar  4 17:21 .
drwxr-xr-x    1 root     root          4096 Mar  4 17:21 ..
-r--r-----    1 root     1000             7 Mar  4 17:21 foo

Is it perhaps a configuration issue, and is it trying to use /run/secrets as file (not directory)?

[hungpham2411]

Is it perhaps a configuration issue, and is it trying to use /run/secrets as file (not directory)?

Looking at the service's log, I don't think so. I will attach it here the exception part: elasticsearch-docker.log
And for addition, the Elasticsearch config:

cluster.name: "docker-cluster"
cluster.initial_master_nodes: es
network.host: 0.0.0.0

## X-Pack settings
## see https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-xpack.html
#
xpack.license.self_generated.type: basic
xpack.security.enabled: true
xpack.security.transport.ssl.verification_mode: certificate 
xpack.security.transport.ssl.key: /run/secrets/elasticsearch-ssl-key 
xpack.security.transport.ssl.certificate: /run/secrets/elasticsearch-ssl-crt
xpack.monitoring.collection.enabled: true

[xeptore]

Docker Compose doesn't support real (swarmkit) secrets, and imitates them by bind-mounting the file directly into the container (which means that permissions on the host are the same as in the container).

@thaJeztah, any chance for contribution in order to somehow fix this issue and make it also work in docker-compose?

[lonix1]

@raratiru @niloct @JustinGuese @wbadart @MohammedNoureldin @Soberia @michitaro I see you upvoted this issue. If this is still important to you, please upvote this new feature request, and/or add your opinions there.