This module expose the Prometheus HTTP server to expose metrics of your Celery backends.
To install celery-prometheus with pip, use the command:
pip install celery-prometheus
With Poetry:
poetry add celery-prometheus
To setup celery-prometheus to your backend, simply call the method add_prometheus_option
after the init of the Celery object.
Example:
from celery import Celery
from celery_prometheus import add_prometheus_option
app = Celery()
add_prometheus_option(app)
# Rest of your code ...Before starting your backend, you will need to expose the PROMETHEUS_MULTIPROC_DIR environment
variable to indicate which folder the Prometheus Client will use to store the metrics
(see Multiprocess Mode (E.g. Gunicorn) of the Promehteus Client documentation).
To start and expose the Prometheus HTTP Server, you need to use the --prometheus-collector-addr
argument when starting your Celery backend:
export PROMETHEUS_MULTIPROC_DIR=/var/cache/my_celery_app
celery worker -A my_celery_backend.backend --prometheus-collector-addr 0.0.0.0:6543Now that your backend is started, you can configure your Prometheus scrappers to scrappe your Celery backend.
This project is open to external contributions. Feel free to submit us a Pull request if you want to contribute and improve with us this project.
In order to maintain an overall good code quality, this project use the following tools:
Linting and formatting tools are configured to match the current default rules of Black.
We also use Mypy as a static type checker.
Please ensure to run these tools before commiting and submiting a Pull request. In case one of these mentionned tools report an error, the CI will automatically fail.
If you're making your first contribution to this project, please add your name to the contributors list.
This project is released by Gandi.net tech team under the BSD-3 license.