Run server entrypoint as non-root

[jarkenau]

Closes #375

[mmeijerdfki]

I tested this locally and the server cannot write to the specified data folders due to filesystem rights. I found the following structure after executing gRPC_pb_sendLabeledImage.py:

docker@a68a8a1ec0dd:/$ ls -alR /mnt
/mnt:
total 12
drwxr-xr-x 1 root root 4096 Mar 22 00:11 .
drwxr-xr-x 1 root root 4096 Mar 22 00:09 ..
drwxr-xr-x 2 root root 4096 Mar 21 23:24 seerep-data

/mnt/seerep-data:
total 8
drwxr-xr-x 2 root root 4096 Mar 21 23:24 .
drwxr-xr-x 1 root root 4096 Mar 22 00:11 ..
-rw-r--r-- 1 root root    0 Mar 21 23:24 logseerep_0.log

Another thing I noticed at

seerep/docker/docker-compose.yml

Line 16 in a2d087b

- SEEREP_DATA_FOLDER=/mnt/seerep-data/

is that the latest / in the path of the SEEREP_DATA_FOLDER could be removed, because the server prints the path as /mnt/seerep-data//4b51b965-62b1-4b23-8bf0-180b7bb0159c.h5.
This just seems to be a cosmetic thing, but would fit in this pr in my opinion.

[jarkenau]

How did you start the server? The volumes section in the docker compose is quite nuanced. The default config uses a named volume, if the mount location does not exist (/mnt/seerep-data) Docker will create it, but owned by root.

However, the comment in:

seerep/docker/docker-compose.yml

Line 13 in a7b33c5

#- /your/local/absolute/path:/mnt/seerep-data #using host folder

states to use a bind mount by providing an absolute path on the host. The permissions are then determined by the permission of the directory. Surprisingly, if the directory on the host does not exist, it will also get created, owned by root.

I totally agree that this default behavior is confusing. I have to think about which volume type makes the most sense for our use case as a named volume is more portable, but with bind mounts we could easily sync HDF5 files to a server directory e.g using rsync.

[mmeijerdfki]

How did you start the server? The volumes section in the docker compose is quite nuanced. The default config uses a named volume, if the mount location does not exist (/mnt/seerep-data) Docker will create it, but owned by root.

However, the comment in:

seerep/docker/docker-compose.yml

Line 13 in a7b33c5

#- /your/local/absolute/path:/mnt/seerep-data #using host folder

states to use a bind mount by providing an absolute path on the host. The permissions are then determined by the permission of the directory. Surprisingly, if the directory on the host does not exist, it will also get created, owned by root.

I totally agree that this default behavior is confusing. I have to think about which volume type makes the most sense for our use case as a named volume is more portable, but with bind mounts we could easily sync HDF5 files to a server directory e.g using rsync.

I started the server using the docker-compose.yml modified to use the locally built image with docker compose up.

As far as I know /mnt on Ubuntu is created by default. If that is the case wouldn't putting RUN chown docker:docker /mnt in the Dockerfile in the section where root would be the executing user fix the issue?

Edit:
I just tested this, but that doesn't work.

[jarkenau]

If that is the case wouldn't putting RUN chown docker:docker /mnt in the Dockerfile in the section where root would be the executing user fix the issue?

Yes, that should generally resolve the problem, however with this approach we would hard code the mount location in the Dockerfile. As a result, we would always have to rebuild the image when the path changes. I'm not sure if this will ever happen, but if it is possible to change the ownership in the docker-compose I think it would be much cleaner.

[jarkenau]

I just tested this, but that doesn't work.

I added this, before switching the user:

RUN mkdir -p /mnt/seerep_data && chown docker:docker /mnt/seerep_data

Resulting in:

docker@56179aa69d92:/mnt/seerep_data$ ll
total 16
drwxr-xr-x 2 docker docker 4096 Mar 22 11:20 ./
drwxr-xr-x 1 root   root   4096 Mar 22 11:17 ../
-rw-r--r-- 1 docker docker 6144 Mar 22 11:20 3faebbd9-7a0b-4575-a4ac-a9f5ed930610.h5
-rw-r--r-- 1 docker docker    0 Mar 22 11:17 logseerep_0.log

Did you delete the old volume?

[mmeijerdfki]

Did you delete the old volume?

Yes, but I set the rights just to the /mnt directory. It seems that you have to change the ownership of the mount directory for it to work correctly as you did.

[jarkenau]

Something like this could work docker/compose#3270 (comment), but it also feels clunky.

[mmeijerdfki]

Something like this could work docker/compose#3270 (comment), but it also feels clunky.

Yeah, this would put the the path configuration for the volume in one place. Additionally environment variables could be used to keep the configuration for the paths in one place. But spinning up another container for that task seems unnecessary.

Maybe something like a .env file could be used for both the docker-compose.yml and the Dockerfile, to centralize configuration in one place? However I didn't research into that topic deeply enough to evaluate it's feasibility.

[jarkenau]

That should work, however if we build the server image in the CI and push it into a registry the path is already pre-determined since it has to be known at build time.

[Mark-Niemeyer]

As our concern is usability and not security right now, we can use this approach.

We should also add an env var in the docker-compose to set the user so that the user in the container and the user of the host can match.

The path within the docker container can be hardcoded. The is no reason why a user would change it.

[jarkenau]

The is no reason why a user would change it.

We once had the idea of using network storage, I thought it could be useful for that.

I now used the approach with a second service for changing the permission. The apline image is around 5 MB and it just takes 1–2 seconds. I also included env variables for the uid and guid.

[mmeijerdfki]

I now used the approach with a second service for changing the permission. The apline image is around 5 MB and it just takes 1–2 seconds. I also included env variables for the uid and guid.

Is there a reason to seperate SEEREP_DATA_FOLDER and SEEREP_LOG_PATH into /tmp and /mnt? Imo putting them together under one root either /tmp or /mnt would make more sense, such that both related data folders can be found in one place.
Otherwise this solution is good I think.

[jarkenau]

Good catch, I forgot to change it back after testing some things.