Deep Extreme Level Set Evolution (DELSE)

This is the PyTorch implementation of DELSE model for object instance segmentation. This repository provides code to train and evaluate with DELSE. For details, please refer to:

Object Instance Annotation with Deep Extreme Level Set Evolution
Zian Wang, David Acuna*, Huan Ling*, Amlan Kar, Sanja Fidler

[Project Page][Paper] [Poster][bibtex]

CVPR 2019

Where is the code?

To get the code, please sign up here. We will be using GitHub to keep track of issues with the code and to update on availability of newer versions (also available on website and through e-mail to signed up users).

If you find this code helpful, please consider citing

@inproceedings{DELSE2019,
title={Object Instance Annotation with Deep Extreme Level Set Evolution},
author={Zian Wang and David Acuna and Huan Ling and Amlan Kar and Sanja Fidler},
booktitle={CVPR},
year={2019}
}

Installation

The code was tested with Anaconda (Python 3.6) and PyTorch 0.4.1.

1. Install dependencies:

   conda install pytorch torchvision -c pytorch
   conda install matplotlib opencv pillow scikit-image
   pip install tensorboard tensorboardx
   

2. Download the pre-trained PSPNet model.

   cd models/
   chmod +x download_pretrained_psp_model.sh
   ./download_pretrained_psp_model.sh

3. Set the paths in mypath.py, so that they point to the appropriate locations of PASCAL/SBD/DAVIS/CityScapes dataset.

Data

Cityscapes

• Download the Cityscapes dataset (leftImg8bit_trainvaltest.zip) from the official website [11 GB]
• Download our processed annotation files from here [68 MB]
• From the root directory, run the following command with appropriate paths to get the annotation files ready for your machine

python dataloaders/change_paths.py --city_dir <path_to_downloaded_leftImg8bit_folder> --json_dir <path_to_downloaded_annotation_file> --output_dir <output_dir>

• Set the path in mypath.py to the location of CityScapes dataset.

Scripts

Project Structure

.
├── networks        				# backend CNN modules
|   └── ...       		
├── layers           			# loss and level set evolution mechanism
|   └── ...
├── dataloaders           			# thank DEXTR and PolyRNN for loaders
|   ├── cityscapes.py       		
|   ├── davis.py        			
|   ├── pascal.py    				
|   ├── sbd.py    					
|   ├── custom_transforms.py    	# transforms for data
|   ├── helpers.py    			# helper functions used in data processing
|   └── ...
├── models				# deeplab pretrained model
|   └── ...
├── evaluation						
|   └── eval.py				# function for eval
|
├── DELSE.py       			# model
├── mypath.py     			# path for datasets
├── eval_multi.py     			# run multi evaluation 
└── main.py        			# training and testing

Training

Run python main.py for training. The default argparse parameters is for Cityscapes. The parameters of settings are in main.py.

Evaluation

Set the index of experiments and evaluation settings in eval_multi.py (Line 14-18 in function param_generateor). Run python eval_multi.py for evaluation on your predicted results.

The model ckeckpoint on Cityscapes dataset can be found here. If the intermediate outputs are not saved, please get the inference outputs first with the test function in DELSE.py.

---

If you have questions with this code, please contact zianwang@cs.toronto.edu. We would like to thank DEXTR for releasing their code.