Mesos Persistent Volumes problems

[elan100cs]

Hi there,

The mesos persistent volumes doesnt work eventhough i added in the marathon run params as instructed here https://mesosphere.github.io/marathon/docs/persistent-volumes.html.

The behavior that im seeing is the data either doesnt get persisted in the panteras container or it gets persisted in the panteras container but doesnt get persisted in the host drive.

After investigating this problems for a few days, i found the problem is due to the docker volume propagation are set to rprivate by default when panteras mapped the mesos work dir.

The rprivate propagation in the docker volume meant that a folder cannot be mounted twice. Since panteras itself is a docker container which mounted the /tmp/mesos workdir, therefore the mesos persistent volumes wont work, because it will also mount /tmp/mesos/volumes directory.

To fix this, we need to set the the propagation of the docker volume to 'shared'. This will allow the folder to be mounted twice within docker volume. You wont find this in the official docker documentation yet. As specified here, the documentation is missing moby/moby#20656

To be specific the fix would be to change the docker-compose.yml.tpl

volumes:
- "/etc/resolv.conf:/etc/resolv.conf.orig"
- "/var/spool/marathon/artifacts/store:/var/spool/store"
- "/var/run/docker.sock:/tmp/docker.sock"
- "/var/lib/docker:/var/lib/docker"
- "/sys:/sys"
- "/tmp/mesos:/tmp/mesos:shared"
- "/tmp/supervisord:/tmp/supervisord"
- "/tmp/consul/data:/opt/consul/data"
- "/proc:/host/proc:ro"
- "/sys:/host/sys:ro"

I am happy to create a pull request if you want me to.

[sielaq]

Since we use mostly stateless appsin PaaS, we didn't notice that. Thanks a lot for info.
We can add this.

[sielaq]

I made it optional, you will have to do
export SHARED=:shared
I can't make it default since volume is not always shared mount (like in travis).

🤔 I should named it differently like MOUNT_OPT

[sielaq]

just for history
#238