Detection of Marine Debris with Sentinel-2

NOTE: The costs of hosting the data and models got too high. I had to set it private. Please contact Marc Russwurm per email if you would like to have access to the data and model weights.

Rußwurm M, Venkatesa J. S., Tuia D. (2023; in submission) Large-scale Detection of Marine Debris in Coastal Areas with Sentinel-2

Getting Started

We provide a python package for easy installation and model predictions

Installation of the Python Package

pip install marinedebrisdetector

# alternative from GitHub directly
pip install git+https://github.com/marccoru/marinedebrisdetector

You now have a marinedebrisdetector program to make predictions of Sentinel-2 tif images on the console. Check marindebrisdetector --help for information on parameters and options.

Predict the qualitative test images from the paper with this command.

marinedebrisdetector --plot-qualitative

predict a Sentinel-2 scene (Durban) with these commands

# download example scene (~300 MB) to ("durban_20190424.tif")
marinedebrisdetector --download-durban

# predict the durban example scene
marinedebrisdetector durban_20190424.tif

the prediction should be in the same folder named durban_20190424_prediction.tif. You can inspect the prediction scene in any image viewer, but best in a GIS program QGIS or ArcGIS that allows you inspect and overlay other georeferenced data.

The tested prediction times of the scene prediction above are:

• 11 sec on NVIDIA GeForce RTX 3090
• 15 sec on NVIDIA GeForce GTX 970
• 11 minutes on Macbook Pro CPU

Pre-trained Models

We provide pre-trained weights for 12-channel Sentinel-2 imagery. A detailed list of weights can be found on the models page

pretrained segmentation models can be loaded via the torch hub in python

import torch

torch.hub.load("marccoru/marinedebrisdetector", "unetpp")
torch.hub.load("marccoru/marinedebrisdetector", "unet")

# trained without label refinement (can lead to thinner more fine-grained predictions)
torch.hub.load("marccoru/marinedebrisdetector", "unetpp", label_refinement=False)

The marinedebrisdetector defaults to the unetpp/unet++.

Datasets

We trained the underlying Unet++ segmentation model (implementation from Pytorch Segmentation Models) on an aggregated a combination of existing datasets for training FloatingObjects Dataset (Mifdal et al., 2020), Marine Debris Archive (Kikaki et al., 2022), S2Ships (Ciocarlan et al., 2021), and newly annotated a refinedFloatingObjects dataset and Sentinel-2 images of the Plastic Litter Projects (Papageorgiou et al., 2022; under review)

More details in a dedicated data page. Executing the training script (see this "model training" page) with --download will automatically download and uncompress the required datasets (116 GB (uncompressed)).

Model Training

We provide a a training script powered by Pytorch Lightning and Weights and Biases to train new models and reproduce our results. More details on training commands and ablations in a dedicated page for model training

Random Forest Comparisons

We compare to a random forest trained on 1) the original MARIDA dataset (Kikaki et al., 2022) and 2) a random forest trained on our dataset. You should be able to reproduce the results in these scripts. The required data and random forest classifier is downloaded automatically.

python comparisons/test_rf_original_data.py
python comparisons/test_rf_our_data.py