Feature request: configurable user and group for writing study_asset and log files

[gmw-web]

Currently the user and group ("daemon:deamon") is hard-coded in the build.sbt file:

https://github.com/JATOS/JATOS/blob/master/build.sbt

Would it be possible to make the user and group more easily configurable somehow?
E.g. by reading them from the Environmental Variables?

Because right now there isn't a Dockerfile that one can easily modify and installing and learning to use the SBT Native Packager/Java for making such a relatively small change is quite a long detour.

I ask this because in our case, the persistent volume storing all the assets is a home-directory that lives on a mounted NFS-share. Each user account/home-directory has a particular disk quota on the NFS, but so does the 'daemon' user, and there's quite a big difference in these quota amounts.

To manage everything properly, both permission and quota-wise, we really would like to make JATOS write files on the NFS home-directory of user 'X' with each file having owner user 'X' and group 'X' instead of deamon/deamon.

Is there any way to make this possible in the future or are we even overlooking a way to accomplish this right now?

[kristian-lange]

I'm very sorry I didn't answer in time. Your issue somehow got lost between Corona and an upgrade to my local family. I only re-discovered it now because I had some time and went through all open issues. Is this still relevant?

[gmw-web]

Is this still relevant?

Yes, it is. It would be great if this could be addressed.

[kristian-lange]

As far as I understand you want JATOS to store the user's study assets in different folders, each user a different folder, e.g.

/home/user1/assets/
/home/user2/assets/
...

Currently that's not possible in JATOS (as you figured out already). Currently JATOS stores everything with the user it was started with, in case it was started in a Docker container, it's the daemon:daemon user. I think it could be implemented, but it's quite difficult.

JATOS would need to know about the existing users in your NFS home directory. This could be stored along each JATOS user in an additional field. Another possibility is that the JATOS user's username and the NFS home user's username are identical. But how would you ensure this?

Then another question is all the other data that belong to a user: the assets are only a part. Usually the study's result data, that are stored in the DB, are the ones that take the most disk space. It's not possible to do something like this within a MySQL database - although we could add quotas per user within JATOS. And then, since a couple of releases, JATOS allows result file uploads from study runs and this feature is more and more used. Those files are stores in even another location (by default in /result_uploads in the JATOS folder).

How about a different idea: I understand, as an JATOS admin, you want to have control over the disk usage of your JATOS users. How about JATOS would offer quota settings for each user in the GUI, in the user manager: you could specify the size for the assets, the DB, and the result file uploads? In the user manager page, each user row would get an additional button 'Quota' which allows to set them and by default they are set to the ones in the production.conf.

Btw., did you notice that the new version 3.6.1 shows already some of the data you want to control. Unfortunately you can't set quotas yet, but at least you know which user is using how much disk space (http://www.jatos.org/Administration.html#study-administration and http://www.jatos.org/User-Manager.html).

Best,
Kristian

[gmw-web]

Dear Kristian,

Thank you for very much your elaborate reply and, in my turn, sorry for the belated reply.

As far as I understand you want JATOS to store the user's study assets in different folders, each user a different folder, e.g.

/home/user1/assets/
/home/user2/assets/
...

Perhaps my explanation wasn't that clear. It wasn't about storing a user's study assets in different folders (the directory structure itself), but about the user:group combination that is used to actually write those files to the filesystem.

Currently, regardless of the study user/owner, each file in the study_assets_root folder looks like this:

drwx-----x+ 5 daemon daemon 4096 Mar 25 08:23 31d1489c75b5d37c629d3f4a29dd74e1
drwx-----x+ 5 daemon daemon 4096 May 14 09:26 c51d64efda85816dc7dc6ac5d63194f5
etc.

So regardless of who the JATOS user is, on the file system everything is owned by daemon:deamon

In our situation, for persistence the entire JATOS study_assets_root directory is an NFS mountpoint.
This NFS mountpoint has its own user, e.g. nfsuser, and that user also has its own disk quota.

What I originally meant was that it would be absolutely ideal if, for example using an Environment Variable, it would be easily possible to change the deamon:deamon user:group combination to a custom user:group combination.

So that it could easily be configured to make the study_assets_root directory like this for example:

drwx-----x+ 5 nfsuser nfsgroup 4096 Mar 25 08:23 31d1489c75b5d37c629d3f4a29dd74e1
drwx-----x+ 5 nfsuser nfsgroup 4096 May 14 09:26 c51d64efda85816dc7dc6ac5d63194f5
etc.

With the user:group combination being the owner of the files in that directory.

Basically, similar to what the Apache webserver does with the APACHE_RUN_USER and APACHE_RUN_GROUP variable to make it run using as an alternate user.

If we could easily change the "run as" user from deamon:deamon to nfsuser:nfsgroup, the study_assets_root folder would be able to adhere to the NFS disk quota that has been set for the nfsuser.

Did the above help to make the feature request more clear?

[kristian-lange]

Ah, I misunderstood you. You want another user and group for JATOS to be able to adhere to disk quota (e.g. nfsuser:nfsgroup).

Why don't you just start JATOS under this different user? Downgrading like with Apache like you suggested is not necessary. JATOS doesn't have to be started as root (or daemon user), any standard user is fine, as long as you don't bind to a port < 1024. The recommended and default way to run JATOS is on port 9000 and then have a reverse proxy (e.g. Apache or Nginx) in front, that binds to port 80 or 443 and forwards the traffic to JATOS. There are example configs for Apache (https://www.jatos.org/JATOS-with-Apache.html) and Nginx (https://www.jatos.org/JATOS-with-Nginx.html) in the documentation. E.g. if you start JATOS with the user and group jatos:jatos, all files (including everything in study_assets_root, result_uploads, and study_logs) will belong to the user j_atos:jatos_. And you can put disk quota on this user.

[gmw-web]

Ah, I misunderstood you. You want another user and group for JATOS to be able to adhere to disk quota (e.g. nfsuser:nfsgroup).

Correct! :)

Why don't you just start JATOS under this different user?

Because JATOS is running as a Docker container right now and the daemon:daemon user is apparently hard-coded in the built.sbt file:

JATOS/build.sbt

Line 30 in 12e82cf

ExecCmd("RUN", "chown", "-R", "daemon:daemon", "."),

Hence my original request: it would be ideal if (e.g. using environmental variables or a config file setting), one could specify the desired "run as" user for JATOS when it's started within the Docker container.

Currently this is apparently not possible. I only see a way to change the database user and password, but not the "run as" user:

https://www.jatos.org/Install-JATOS-via-Docker.html
https://www.jatos.org/Configure-JATOS-on-a-Server.html

[kristian-lange-tw]

Ah, you are using Docker ... I missed this one.

In Java or the JVM it's not easy / impossible to change the user the program it is running with. To change users (like Apache it's doing) Java would have to do system calls and they are different from OS to OS.

But if you are up to it you could create your own dockerfile. You probably would just change the 'daemon' user and group.

[gmw-web]

Do I understand correctly that:

• JATOS is using the daemon user and group, which is set in the build.sbt file.
• This user is used because the JATOS Docker environment is based on the OpenJDK image:
  

  JATOS/build.sbt

  Line 23 in 12e82cf

  Cmd("FROM", "openjdk:11-jre"),

• This openjdk:11-jre image has a daemon user, which is the one that is used

?

If so, probably the fastest route would be to try and add a create user command to the build.sbt file, within the dockerCommands sequence. Otherwise I'd have to look at recreating a custom OpenJDK image as well.

I have never done anything with .sbt files though. Could you perhaps shortly explain to me, or point me to a relevant tutorial, how one uses a .sbt file to create a Docker image? I'm only used to writing a Docker/Docker compose file directly.

How do you normally create the JATOS Docker image e.g. for a new release? Thanks :)

[kristian-lange-tw]

Sorry for replying so late.

JATOS is using the daemon user and group, which is set in the build.sbt file.

Yes

This user is used because the JATOS Docker environment is based on the OpenJDK image

Yes

This openjdk:11-jre image has a daemon user, which is the one that is used

Yes

If you just want to try it out I can create the docker image for you and push it to Docker Hub. It wouldn't be to much work for me since my system is all set up.

If you want to do it yourself I can see 2 ways how you can achieve a docker image with JATOS running under a specified user:

1. You pull a fresh openjdk:11-jre docker image, run a shell in there and install JATOS like you would on any other Linux system and use the user you want to run JATOS. And you don't have to use the openjdk:11-jre image - you can use any Linux image to start with. But then you have to install Java in the container additionally.

2. Pull the JATOS repo from GitHub and install sbt in version 0.13. Then change in /build.sbt, the section dockerCommands to use your user. It should look something like this (I didn't actually try it):

   dockerCommands := Seq(
     Cmd("FROM", "openjdk:11-jre"),
     Cmd("WORKDIR", "/opt/docker"),
     Cmd("ADD", "opt /opt"),
     Cmd("EXPOSE", "9000 9443"),
     Cmd("RUN", "apt update -y && apt install vim -y"),
     ExecCmd("RUN", "mkdir", "-p", "/opt/docker/logs"),
     ExecCmd("RUN", "addgroup", "-g", "1001", "-S", "appuser"),
     ExecCmd("RUN", "adduser", "-u", "1001", "-S", "appuser", "-G", "appuser"),
     ExecCmd("RUN", "chown", "-R", "appuser:appuser", "."),
     ExecCmd("RUN", "chmod", "u+x", "loader.sh"),
     Cmd("VOLUME", "/opt/docker/logs"),
     Cmd("RUN", "bash -l -c 'echo export JATOS_SECRET=$(LC_ALL=C tr -cd '[:alnum:]' < /dev/urandom | fold -w128 | head -n1) >> /etc/bash.bashrc'"),
     Cmd("USER", "appuser"),
     ExecCmd("ENTRYPOINT", "./loader.sh", "start")
   )
   

   With sbt I use a plugin (https://www.scala-sbt.org/sbt-native-packager/formats/docker.html) to create the docker image: run sbt docker:publishLocal in the terminal and your docker image will be created.