A deep learning framework to help you build your species distribution models
View framework · Report Bug · Request FeatureMalpolon is a framework facilitating the training and sharing of Deep Species Distribution models using various types of input covariates including bioclimatic rasters, remote sensing images, land-use rasters, etc...
If you're not a deep learning or PyTorch expert but nonetheless want to use visual deep learning models on satellite and/or bioclimatic rasters to predict the presence of species on a given area, this framework is for you.
Malpolon is destined to be used by various user profiles, some more experimented than others. To this end, we provide several examples of usage of the framework, organized by use case or scenarios. These examples can be found in the examples folder of the repository, each with a README file for more details on how to use the scripts.
Here is a list of the currently available scenarios:
- Kaggle : I am a potential kaggle participant on the GeoLifeClef challenge. I want to train a model on the provided datasets without having to worry about the data loading, starting from a plug-and-play example.
- GeoLifeClef2022 : contains a fully functionnal example of a model training on the GeoLifeClef2022 dataset, from data download, to training and prediction.
- GeoLifeClef2023 : contains dataloaders for the GeoLifeClef2023 dataset (different from the GLC2022 dataloaders). The training and prediction scripts are not provided.
- Ecologists : I have a dataset of my own and I want to train a model on it. I want to be able to easily customize the training process and the model architecture.
- Drop and play : I have an observations file (.csv) and I want to train a model on different environmental variables (rasters, satellite imagery) without having to worry about the data loading.
- Custom dataset : I have my own dataset consisting of pre-extracted image patches and/or rasters and I want to train a model on it.
- Inference : I have an observations file (.csv) and I want to predict the presence of species on a given area using a model I trained previously and a selected dataset or a shapefile I would provide.
- Benchmark : I want to compare the performance of different models on a given known dataset, or validate the framework's backbone (every module apart from the datasets and datamodule) on a specific hardware.
To install malpolon, you will first need to install Python ≥ 3.10, and several python packages. To do so, it is best practice to create a virtual environment containing all these packages locally.
Click here to expand instructions
Before proceeding, please make sure the following packages are installed on your system:
- Python ≥ 3.10
pipgitlibgeos-dev(dependency of Python libraryCartopy)- On Linux (Ubuntu):
sudo apt install libgeos-dev - On MacOS:
brew install geos
- On Linux (Ubuntu):
cmake- On Linux (Ubuntu):
sudo apt install cmake - On MacOS:
brew install cmake
- On Linux (Ubuntu):
cuda(if you intend to run your models on GPU)
The following instructions show installation commands for Python 3.10, but can be adapted for any of the compatible Python versions metionned above by simply changing the version number.
The backend side of malpolon is distributed as a package on PyPi. To install it, simply run the following command:
pip install malpolon
However, versions available on PyPi are non-experimental and possibly behind the repository's main and dev branches. To know which version you want download, please refer to the tags section of the repository and match it with PyPi.
Furthermore, the PyPi package does not include the examples and the documentation. If you want to install the full repository, follow the next steps.
Clone the Malpolon repository using git in the directory of your choice:
git clone https://github.com/plantnet/malpolon.git
- Via
virtualenv
We recommend handling your virtual environment using virtualenv (or similar) and installing the packages via pip.
First create your virtual environment using the proper python version, and activate it (note that in this example, the virtual environment "malpolon_env" will be installed in the current directory).
virtualenv -p /usr/bin/python3.10 ./malpolon_3.10
source ./malpolon_3.10/bin/activate
Once the env is activated, install the python packages listed in requirements_python3.10.txt:
pip install --upgrade setuptools
pip install -r requirements_python3.10.txt
- Via
conda
You can also use conda to install your packages.
conda env create -f environment_python3.10.yml
conda activate malpolon_3.10
The malpolon repository can also be installed in your virtual environment as a package. This allows you to import malpolon anywhere in your scripts without having to worry about file paths. It can be installed via pip using:
cd malpolon
pip install -e .
To check that the installation went well, use the following command
python -m malpolon.check_install
which, if you have CUDA properly installed, should output something similar to
Using PyTorch version 2.1.0+cu121
CUDA available: True (version: 12.1)
cuDNN available: True (version: 8902)
Number of CUDA-compatible devices found: 1
The dev branch is susceptible to have more up-to-date content such as newer examples and experimental features. To switch to the dev branch locally, run:
git checkout dev
An online code documentation is available via GitHub pages at this link. This documentation is updated each time new content is pushed to the main branch.
Alternatively, you can generate the documention locally by following these steps :
- Install the additional dependences contained in
docs/docs_requirements.txtmust be installed
pip install -r docs/docs_requirements.txt
- Generated the documentation
make -C docs html
The result can be found in docs/_build/html.
This roadmap outlines the planned features and milestones for the project. Please note that the roadmap is subject to change and may be updated as the project progress.
Click here to toggle roadmap
- Data support
- Images (pre-extracted patches)
- Rasters
- Time series
- Via GLC23 loaders (.csv)
- Via generic loader
- Shapefiles
- Fuse several data types in one training
- Deep learning tasks
- Binary classification
- Multi-class classification
- Multi-label classification
- Regression (abundance prediction)
- Ordinal
- Supported models
- CNN
- LSTM
- Transformers
- Training flexibility
- Add model head/tail modifiers
- Allow easy (un-)freeze of layers
- Allow dataset intersections and unions
- Allow data parallel training
- Multithreading
- Multiprocessing
- Issues may arise depending on hardware
Here is an overview of the main Python librairies used in this project.
- To handle deep learning loops and dataloaders
- Deep learning framework which simplifies the usage of PyTorch elements
- For common computational operations
- To handle data rasters
- For displaying purposes
- To handle models' hyperparameters
- To handle geographical data
This work is made possible through public financing by the European Commission on european projects MAMBO and GUARDEN.
This work is currently under development and maintained by the Pl@ntNet team within the INRIA research institute.
This framework is ditributed under the MIT license, as is the Pl@ntNet project. See LICENSE.md for more information.
Malpolon is a project developed by the Pl@ntNet team within the INRIA research institute. If you use this framework in your research, please cite this repository in your paper.
Authors include :
- Théo Larcher (current lead developper) (email)
- Maximilien Servajean
- Alexis Joly
Former developpers include :
- Titouan Lorieul (former lead developper) (email)