Volume owned by root

[keshav-dataco]

Bug description

Below are lines from my jupyter_config.py

c.DockerSpawner.volumes = { 
'jupyterhub-user-{username}':'/home/jovyan/work',
'jupyterhub-shared': {"bind": '/home/jovyan/shared', "mode": "ro"}
}

/home/jovyan/work is owned by jovyan as expected

Expected behaviour

/home/jovyan/shared is owned by jovyan

Actual behaviour

/home/jovyan/shared is owned by root

How to reproduce

Spin up a user container with above config

Your personal set up

• OS:
• ubuntu 20.04
• docker latest
• latest jupyterhub and dockerspawner

Any help would be much appreciated

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[minrk]

Maybe this has changed, but I think it's up to the docker volume provider to set initial permissions. We don't expose volume creation options in DockerSpawner, yet.

I believe the work ownership is set by the fact that the directory exists in the image, while shared doesn't so it uses the default of root:root.

If the empty shared mount point exists in your image with the right ownership, it may work correctly.

Alternately, you can manage creation of the shared volume and its ownership/permissions outside jupyterhub. I assume you are doing this already, since if it's mounted read-only there must be something else with write access to put things there.

Since it's read-only, is there an issue with it being owned by root?

[keshav-dataco]

Thank you so much @minrk for quick response. I was lazy and 'ro' was copied from other issue. I tried 'rw' too.

I will follow your lead and see if i can have the shared directory present with right perms

[matercomus]

I have the same issue. The only difference is that I have the shared folder created inside of work so it shows up in explorer by default.
c.DockerSpawner.volumes = { 'jupyterhub-user-{username}':'/home/jovyan/work', 'jupyterhub-shared': {"bind": '/home/jovyan/work/shared', "mode": "rw"} }
But even with "mode": "rw" I still cannot access it from the spun-up container. Any help would be appreciated!

[matthewwiese]

In the interest of visibility, I also encounter this problem. The related part of my config:

notebook_dir = "/home/jovyan/work"
c.DockerSpawner.notebook_dir = notebook_dir
c.DockerSpawner.volumes = { "/mnt/volume_nyc3_01/hub_userdata/{username}": notebook_dir }

The absolute path is meant to mount from the Docker host, which it does correctly; I am running rootless Docker and as such the directories created above are owned by my system user.

However, in spite of this, the directory in the container is owned by root:

(base) jovyan@87cdb8904abf:~$ ls -lh
total 4.0K
drwxr-xr-x 2 root root 4.0K Jul 18 15:54 work

I have managed to find relevant issues elsewhere on GitHub, albeit none with a solution that works in my case:

• Default folder recreates with root owner #127
  • In particular, this suggestion doesn't resolve the problem
• Docker volumes permission denied jupyterhub-deploy-docker#86

[matthewwiese]

Following on my previous comment, I want to update with additional information.

I've been able to reproduce this problem - with an absolute path mount location - on a vanilla Ubuntu 22.04 VM. The files are at this Gist.

Based on my reading of the documentation, the following should work:

notebook_dir = "/home/jovyan/work"
c.DockerSpawner.notebook_dir = notebook_dir
c.DockerSpawner.volumes = { "/tmp/{username}": notebook_dir }

To mount a directory for some user (e.g. /tmp/foo for user foo) into the container at /home/jovyan/work, which is a directory that already exists in the Jupyter Docker Stacks.

Like my previous comment, the user lacks permissions to write to this mounted directory, and at least the UID/user reported from within a JupyterLab terminal in the spawned container reports 0 0/root root.

My purpose for this is obviously data persistence/permanent storage, so the fact that the user has no write access is behavior I would consider a bug, as what use is the permanent storage if one can't use it? My expectation is that the above jupyterhub_config.py snippet ought to work out-of-the-box, because it matches my expectations for how mounts work elsewhere in the Docker ecosystem (e.g. Compose files).

I've been able to work around this issue with the following change:

c.DockerSpawner.volumes = { "{username}": notebook_dir }

Which creates a Docker volume that is writable, and the reported permissions from within the container are 1000 100/jovyan users. However, I lose the ability to control the location where files are stored unless I create a symlink like this StackOverflow post.

My question for the maintainer(s): is what I encounter expected behavior (i.e. absolute paths causing write permission errors)? If so, could clarity be added to the documentation?

cc @minrk @consideRatio @manics

[kmahyyg]

Same problem here. If there is a new user registered, and volume name is formatted with username and bind to a folder in local disk, I assume this situation is very common. And personally, I recommend to automatically change the folder owner with a helper program or something else.

I did the following things:

Config:

c.JupyterHub.spawner_class = 'dockerspawner.DockerSpawner'
notebook_dir = os.getenv("DOCKER_NOTEBOOK_DIR") or '/home/jovyan/work'
c.DockerSpawner.notebook_dir = notebook_dir
c.DockerSpawner.volumes = { 
    '/userdata/isolation/{username}': notebook_dir
}
c.DockerSpawner.read_only_volumes = {
    '/userdata/public': '/home/jovyan/public',
}

Workaround:

root@debian:~/jobs# cat ./fixperm-isolation.sh
#!/bin/bash
chown -R 1000:1000 /userdata/isolation
root@debian:~/jobs# crontab -l
# Edit this file to introduce tasks to be run by cron.
# For more information see the manual pages of crontab(5) and cron(8)
#
# m h  dom mon dow   command
*/2 * * * * bash /root/jobs/fixperm-isolation.sh

The script mentioned above must be set to 0755 and only root can operate.
This is just a temporary workaround.

A better way for this, without GRANT_SUDO=yes (which break the least privilege rule, especially for untrusted users, my base docker image is scipy-notebook from jupyter docker-stacks), may be write a helper program and write a new document in FAQ to tell administrators to customize docker notebook image and let this helper run under root after container has been started. However, this will lead to a new daemon in container.

How about your ideas? @minrk @consideRatio @manics @matercomus @keshav-1835i

[Falven]

Maybe this has changed, but I think it's up to the docker volume provider to set initial permissions. We don't expose volume creation options in DockerSpawner, yet.

I believe the work ownership is set by the fact that the directory exists in the image, while shared doesn't so it uses the default of root:root.

If the empty shared mount point exists in your image with the right ownership, it may work correctly.

Alternately, you can manage creation of the shared volume and its ownership/permissions outside jupyterhub. I assume you are doing this already, since if it's mounted read-only there must be something else with write access to put things there.

Since it's read-only, is there an issue with it being owned by root?

This would not work if the volume you are mounting is non-empty.

I am having this issue as well. Something somewhere is broken, because It is my understanding that setting:

extraPodConfig:
    securityContext:
      runAsUser: 1000
      runAsGroup: 100
      fsGroup: 100

Should make mounted volumes owned by 1000. The only thing that is working here is fsGroup setting the mounted volume group to 100.