media folder not accessible when not in debug mode

[robe2]

When I run my pretalx with debug=false

/media/whatever always resolves to a 404, though the /static/ works fine.

I don't think the /static is reading from the right folder though, cause if I remove a file from where it should be, it still seems to read the file fine.

For my docker-compose.yml, I have this:
The docker image, was built using the docker recipe in this repo + adding pretalx-public-voting and pretalx-public-pages plugins

version: '3'
services:
  pretalx:
    image: docker.osgeo.org/sac/pretalx-osgeo:latest
    container_name: pretalx
    restart: unless-stopped
    depends_on:
      - redis
      - db
    environment:
       PRETALX_FILESYSTEM_MEDIA: /data/media
       PRETALX_FILESYSTEM_STATIC: /data/static
    ports:
      - "8355:80"
    volumes:
      - ./conf/pretalx.cfg:/etc/pretalx/pretalx.cfg:ro
      - pretalx-data:/data
    labels:
       traefik.docker.network: "pretalxdocker"
       traefik.enable: "true"
:

volumes:
  pretalx-db:
  pretalx-data:
  pretalx-redis:

What I do notice is when I upload a file to an event, it does correctly go into the pretalx-data volume (even when debug=false), so it seems to be just the web url mapping that is off.

I also tried changing from docker volumes to just folder mounts and had the same issue.

[rixx]

pretalx doesn't serve files in /media, FWIW, that's the job of the webserver of choice (in this case traefic, apparently).

No idea how this shakes out with Docker and volumes etc, just as a hint to go error hunting. Happy to merge a PR.

[robe2]

pretalx doesn't serve files in /media, FWIW, that's the job of the webserver of choice (in this case traefic, apparently).

No idea how this shakes out with Docker and volumes etc, just as a hint to go error hunting. Happy to merge a PR.

I'm running nginx as proxy, but on a different server, I tried removing those labels for traefik, which were copied from the boilerplate docker-compose.yml, but that sadly didn't help.

My nginx is pretty much boiler plate described here:

https://github.com/pretalx/pretalx-docker/blob/main/reverse-proxy-examples/nginx/Readme.MD

Except instead of localhost, it's the private network name of the server I have pretalx housed on.

I'm assuming it's something in the GUnicorn setup. Isn't that what is running as the internal webserver.

Neither the traefik or the nginx instructions, make mention of explicitly setting the location of the https://server/media and https://server/static paths.

I had assumed this was handled by:

PRETALX_FILESYSTEM_MEDIA: /data/media
PRETALX_FILESYSTEM_STATIC: /data/static

FWIW, this all used to work fine in pretalx v2.2.0 version which I'm upgrading from.
The main difference I see between the docker-compose of then and now is the introduction of the these new system variables, and removing them from the pretalx.cfg file.

[rixx]

Yeah, this repo is entirely community-maintained and not officially recommended, as I have no interest in Docker, so I just merge all PRs this repo gets (that don't look obviously malicious). pretalx has never served files under /media on its own, that was always up to the web server, so some of the changes must have broken this. (Making me think it may be time to remove even the link to this repo from the official docs, tbh – probably better to not mention this repo at all rather than having people get frustrated with it not working.)

[robe2]

Okay I found the commit that did it. I guess before nginx was included in the image.

702197a

there probably should be some documentation around this, or have a docker-compose that launches an nginx or traefik container that people can remove if they so choose.

[rixx]

Happy to merge a PR to that end.

[robe2]

Happy to merge a PR to that end.

I whipped up a docker-compose that includes an nginx container (and restores the nginx.conf file) so it behaves like it used to.
I'll be happy to provide a PR for that, but probably better as a template or a commented out section in the Dockerfile.

My thought is just have it as a commented out section.

But it feels a little like overkill, in my case since I'm just going to proxy that nginx to my production nginx that handles the certs, but my webserver is on a separate server so doesn't have access to the folders or docker volumes of this server.

What's the reason why media can never be served except when in debug mode?
I was thinking any concerns about access would be satisfied with the GUNICORN_FORWARDED_ALLOW_IPS but that didn't seem to work either.

If I am understanding correctly that the nginx and traefik.toml need to serve the /media files directly, then the instructions in reverse-proxy-examples/docker-compose and reverse-proxy-examples/nginx/Readme.MD are out of date as well, since they don't have a section for the /media folder

[rixx]

They aren't so much "out of date" as straight up wrong, as pretalx has always behaved like that. We follow the Django project guidance on this issue, which is to not use Django to serve user-uploaded files.

[robe2]

They aren't so much "out of date" as straight up wrong, as pretalx has always behaved like that. We follow the Django project guidance on this issue, which is to not use Django to serve user-uploaded files.

In the 2.3, it was right, because internally there was an nginx running as part of the pretalx image, and you can see from that, that it did do as you described, specifically serve up the media, static and then delegate the other to GUNICORN which it had running on a unix socket.

https://github.com/pretalx/pretalx-docker/blob/v2.3.2/deployment/docker/nginx.conf#L48

But all that I think became invalid as soon as the nginx in the container was removed to save space.

My plan is to basically restore this file, but have it mount to the nginx instance I am adding to the docker compose.

So my revised docker-compose will look like this:

version: '3'
services:
  pretalx:
    image: pretalx/standalone:latest
    container_name: pretalx
    restart: unless-stopped
    depends_on:
      - redis
      - db
    environment:
       PRETALX_FILESYSTEM_MEDIA: /data/media
       PRETALX_FILESYSTEM_STATIC: /data/static
    volumes:
      - ./conf/pretalx.cfg:/etc/pretalx/pretalx.cfg:ro
      - pretalx-data:/data
:
  web:
    image: nginx:latest
    ports:
      - "80:80"
    depends_on:
      - pretalx
    volumes:
      - ./deployment/docker/nginx.conf:/etc/nginx/nginx.conf:ro
      - pretalx-data:/data


volumes:
  pretalx-db:
  pretalx-data:
  pretalx-redis:

and then changing this line https://github.com/pretalx/pretalx-docker/blob/v2.3.2/deployment/docker/nginx.conf#L61

to

proxy_pass http://pretalx:80;

I think similar could be done with traefik for those who'd prefer to get their cert etc as part of the docker compose launch, but I'm less familiar with traefik and have a dedicated nginx server on the private network that handles the certs already.

80:80 of the nginx container could be changed to 8355:80 if people have another local nginx that is monitoring their certs already running on 80.

[almereyda]

It is generally good practice to run separate containers for separate processes and even better to only run a single process per container.

What used to be the standalone container is now what would often we called the app container, while Nginx serves as web container that does all static hosting and forwarding to the app for appropriate routes. This comes with the need of a shared volume for Django's static assets regenerated during each migration.

I will be happy to review a PR, when it is filed, or to contribute one during the next setup that's ongoing until May first.

I can second many ideas and arguments from the discussion and the recent commits. Though I'm a bit surprised the setup is listening on port 80 of the host. Most probably there is going to be another reverse proxy, very often Traefik, that acts as web and webs endpoint. I think we can safely assume the Nginx/web container to be exposed to an internal web network only, while carrying just enough Traefik labels to document a complete live environment.

We can provide different compose.yaml, compose.live.yaml, compose.dev.yaml examples to the repository to show how things can be adapted. Because we already see that the expectation of a live deployment is not shared by everyone, as #43 by @MikkCZ clearly states and implies to support the development use case first.

We can also compare the practices here a bit against how other major Django projects do it. Pretix, Authentik, Mailman or FidusWriter come to mind.

And I agree that this repository is in some kind of messorganic shape. What are the Ansible folders for, for example? Happy to find them refactored into a separate repository later on.

It will also be good to revise the README with a documentation of the whole lifecycle of the application, including eventual intermediary steps during migrations.

Right now the repository is in an opinionated and underdocumented state, which we can gradually improve at the different places suggested above. More on Sunday.

[rixx]

I wasn't aware how broken this repository is at the moment, tbh, and it's looking a bit like "community-maintained" doesn't really work.

I will observe the situation for a while longer (say another three months, until ~August), and if the repository is still not in a usable state, I think it would be best to archive it. There are already at least three other Docker setups for pretalx up here on GitHub, so it's not like we'd be leaving people without anything to build on – and it would presumably be less frustrating to have to figure something out, than to try a repo that is (despite big warnings) hosted under the official pretalx org, only to find out it doesn't work. Or, as is the case here, only works with insecure settings, which is even worse, as it encourages people to run pretalx in development mode.

[DASPRiD]

I don't think that this is actually a bug. Pretalx itself does say that media (and preferably static) should be served by a separate application/container. This is really easy to do (e.g. with https://hub.docker.com/r/halverneus/static-file-server).

So I don't think that this repository is actually broken, beside the issue with env forwarding (for which I opened a PR already).

[DASPRiD]

Also I want to note: all three linked docker setups are working the same way, where they don't serve the media directory themself either.