Add support for falcon stack

[pbenini]

No description provided.

[what-the-diff]

PR Summary

• Introduction of New Python Packages
  This PR introduces three new Python packages to our codebase. The packages are falcon 3.1.1, uvicorn 0.24.0.post1, and gunicorn 21.2.0. These packages are key tools for building and running modern web applications more effectively, leading to better performance and increased user satisfaction.

[philipkozeny]

Can you tell us more about the PR and why you propose adding those packages in a specific version?

[pbenini]

Sure.

My propose is to add this package to enable support for python frontend by falcon.

Nominatim supports both php frontend and python frontend .

I did some tests on my local environment and i found that your image is already configured to start python api except for those three modules.

With theese 3 modules you can run nominatim service by running:

export PYTHONPATH="/usr/local/lib/nominatim/lib-python/"
gunicorn -b 127.0.0.1:80 -w 4 -k uvicorn.workers.UvicornWorker nominatim.server.falcon.server:run_wsgi 

or, of course, -b unix:/var/run/nominatim/nominatim.sock if you prefere serve it by nginx through unix socket.

I've add this package to this specific version because is the version I tested in my kubernetes environment.

[leonardehrenfried]

I must admit I don't quite understand what is going on here so that's what I'm assuming:

• atm nominatim has a php and a python frontend
• PHP will be replaced by python in the long run
• therefore it's a good idea to allow people to experiment with the python one

If this is the case, I'm in favour of merging this.

@lonvia Do you think that installing these packages in the container is a good idea?

[lonvia]

Yes, that's a good idea. The Python frontend is currently still experimental but will become the default in the next version and the only available one by the end of the year. See https://nominatim.org/2023/12/19/roadmap-nominatim5.html for the timeline. There is plenty of time for you to move the container to Python but it's never to soon to start experimenting and report issues.

The latest version of these packages should usually be fine.

[jpylypiw]

Hi there!
I just "implemented" the new python frontend with the docker image. To use the python frontend I customized the start.sh script and edded the following:

# fork a process and wait for it
tail -Fv /var/log/postgresql/postgresql-14-main.log /var/log/apache2/access.log /var/log/apache2/error.log /var/log/replication.log &
tailpid=${!}

# custom python api implementation
pip3 install --no-input falcon uvicorn gunicorn
sudo PYTHONPATH="/usr/local/lib/nominatim/lib-python/" -u nominatim gunicorn -b 0.0.0.0:81 -w 4 -k uvicorn.workers.UvicornWorker nominatim.server.falcon.server:run_wsgi &
pythonpid=${!}
# custom python api implementation end

export NOMINATIM_QUERY_TIMEOUT=600
export NOMINATIM_REQUEST_TIMEOUT=3600

at the start I extended the stopServices function

tailpid=0
replicationpid=0
pythonpid=0

stopServices() {
    service apache2 stop
    service postgresql stop
    kill $replicationpid
    kill $tailpid
    kill $pythonpid
}

It is important to execute the python api as nominatim user sind the database connection only can be established as nominatim user.
In my example the new python frontend is listening on port 81 so this is not conflicting with the normal port 80 or 8080.
Maybe this is helpful for someone who is trying around with the new frontend.

[leonardehrenfried]

Thanks for the instructions @jpylypiw .

@philipkozeny Can we merge this?

[leonardehrenfried]

If the python frontend is the future I think we should support/enable it at some stage.

[lonvia]

It is important to execute the python api as nominatim user sind the database connection only can be established as nominatim user.

The python API should be run under the user 'www-data', just like the PHP frontend. This is a user with read-only rights and can do less damage.

[philipkozeny]

Thanks for adding the modules. I suggest changing to minimum required versions instead of specific ones. Currently, setting specific versions means we have to constantly monitor updates and security fixes and manually update both the version and the code.

I also suggest we create a follow-up PR to properly support the new Python frontend.

[stouch]

Hi, following this thread osm-search/Nominatim#3154 (reply in thread), I ask here if there is something new about this issue ? Thanks !

[leonardehrenfried]

We want to implement it but haven't done so. Do you want to send a PR?

[stouch]

Oh , I'm not sure at all to be able to help on this sorry.. I was just hoping for this PR after @lonvia told me about this issue 😇