Docker install: What's the correct way to store the media directory outside the mediacms home?

[DecaTec]

Due to storage requirements, I want to store all the media files (folder media_files) outside the mediacms home on a Docker install. E.g. the media files should be stored on a mounted NFS share (mountet as /mnt/nfs/mediacms/media_files.

I've now tried it with modifying the docker-compose.yaml (see the added volumes for 'web' and 'celery_worker'):

version: "3"

services:
  migrations:
    image: mediacms/mediacms:latest
    volumes:
      - ./:/home/mediacms.io/mediacms/
    environment:
      ENABLE_UWSGI: 'no'
      ENABLE_NGINX: 'no'
      ENABLE_CELERY_SHORT: 'no'
      ENABLE_CELERY_LONG: 'no'
      ENABLE_CELERY_BEAT: 'no'
    depends_on:
      redis:
        condition: service_healthy
      db:
        condition: service_healthy
  web:
    image: mediacms/mediacms:latest
    deploy:
      replicas: 1
    ports:
      - "8008:80"
    volumes:
      - ./:/home/mediacms.io/mediacms/
      - /mnt/nfs/mediacms/media_files/:/home/mediacms.io/mediacms/media_files
    environment:
      ENABLE_CELERY_BEAT: 'no'
      ENABLE_CELERY_SHORT: 'no'
      ENABLE_CELERY_LONG: 'no'
      ENABLE_MIGRATIONS: 'no'
    depends_on:
      - migrations
  celery_beat:
    image: mediacms/mediacms:latest
    volumes:
      - ./:/home/mediacms.io/mediacms/
    environment:
      ENABLE_UWSGI: 'no'
      ENABLE_NGINX: 'no'
      ENABLE_CELERY_SHORT: 'no'
      ENABLE_CELERY_LONG: 'no'
      ENABLE_MIGRATIONS: 'no'
    depends_on:
      - redis
  celery_worker:
    image: mediacms/mediacms:latest
    deploy:
      replicas: 1
    volumes:
      - ./:/home/mediacms.io/mediacms/
      - /mnt/nfs/mediacms/media_files/:/home/mediacms.io/mediacms/media_files 
    environment:
      ENABLE_UWSGI: 'no'
      ENABLE_NGINX: 'no'
      ENABLE_CELERY_BEAT: 'no'
      ENABLE_MIGRATIONS: 'no'
    depends_on:
      - migrations
  db:
    image: postgres
    volumes:
      - ../postgres_data:/var/lib/postgresql/data/
    restart: always
    environment:
      POSTGRES_USER: mediacms
      POSTGRES_PASSWORD: mediacms
      POSTGRES_DB: mediacms
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U mediacms"]
      interval: 10s
      timeout: 5s
      retries: 5
  redis:
    image: "redis:alpine"
    restart: always
    healthcheck:
      test: ["CMD", "redis-cli","ping"]
      interval: 30s
      timeout: 10s
      retries: 3

This seems to work, but I still have a few questions:

• Is there a better/official way to achieve this?
• Do only the containers 'web' and 'celery_worker' need access to the media_files directory or should the corresponding volume be added to the other containers too?

And another offtopic question:
Why the volume for PostgreSQL is not inside the working dir of mediacms?

That's part of the docker-compose.yaml:

- ../postgres_data:/var/lib/postgresql/data/

Wouldn't this be sufficient:

- ./postgres_data:/var/lib/postgresql/data/

[swiftugandan]

I would treat the docker-compose files here as basic examples. You should be able to adjust them to suit your own needs.

• What issues are you experiencing specifically after your changes?

[klausmcm]

@swiftugandan I found this thread after trying something similar but in my case I first tried to specify an absolute path to another folder for volumes. For example, I changed

    volumes:
      - ./:/home/mediacms.io/mediacms/

to

    volumes:
      - /home/user/media:/home/mediacms.io/mediacms/

and when I try to deploy with docker-compose up, I get the following error

ERROR: for ms8e_celery_beat  Cannot start service ms8e_celery_beat: b'OCI runtime create failed: container_linux.go:344: starting container process caused "exec: \\"./deploy/docker/entrypoint.sh\\": stat ./deploy/docker/entrypoint.sh: no Creating ms8e_migrations  ... error

ERROR: for ms8e_migrations  Cannot start service ms8e_migrations: b'OCI runtime create failed: container_linux.go:344: starting container process caused "exec: \\"./deploy/docker/entrypoint.sh\\": stat ./deploy/docker/entrypoint.sh: no such file or directory": unknown'

ERROR: for ms8e_celery_beat  Cannot start service ms8e_celery_beat: b'OCI runtime create failed: container_linux.go:344: starting container process caused "exec: \\"./deploy/docker/entrypoint.sh\\": stat ./deploy/docker/entrypoint.sh: no such file or directory": unknown'

ERROR: for ms8e_migrations  Cannot start service ms8e_migrations: b'OCI runtime create failed: container_linux.go:344: starting container process caused "exec: \\"./deploy/docker/entrypoint.sh\\": stat ./deploy/docker/entrypoint.sh: no such file or directory": unknown'

I renamed the containers by prefixing with ms8e. No issues when using the default settings for volumes.

[klausmcm]

I should also mention that deployment works without issues when using named volumes.

[falense]

Found this about a year later. The problem is actually quite simple. The example compose file expects the source to be available at the location you are mounting. Essentially you are overwriting/mounting over the file the image need to boot. Remove the volumes altogether and set up more specific mounts to resolve this. I.e. just for media, databases etc. The essential files for the docker image image to start is found in /home/mediacms.io/mediacms/deploy

[gimail]

How did you solve the local disk access problem on Windows for Docker. Can you tell me how you did it? I did as recommended above:

• /mnt/mediacms/:/home/mediacms.io/mediacms/media_files.
  But I can’t do it and get the error:
  File "/home/mediacms.io/lib/python3.11/site-packages/django/core/files/storage/filesystem.py", line 130, in _save
  os.chmod(full_path, self.file_permissions_mode)
  PermissionError: [Errno 1] Operation not permitted: '/home/mediacms.io/mediacms/media_files/chunks/3b3239cc-95ea-4aac-bf4b-36d528e5b25b/1_QH7SH3u'.
  I connected a folder on the disk for WSL, but although Docker saves the files, further processing results in an error.

I would appreciate your answer

[DecaTec]

The solution might only work on Linux, as the os.chmod doesn't work on Windows.
But your media files are on Windows and MediaCMS (or Django) tries to chmod these from inside the Docker container.

[supraxd2021]

is it work ? only add nfs volumes in docker-compose.yaml?