CRF-RNN for Semantic Image Segmentation - Keras/Tensorflow version

Live demo:      http://crfasrnn.torr.vision
Caffe version: http://github.com/torrvision/crfasrnn

This repository contains Keras/Tensorflow code for the "CRF-RNN" semantic image segmentation method, published in the ICCV 2015 paper Conditional Random Fields as Recurrent Neural Networks. This paper was initially described in an arXiv tech report. The online demo of this project won the Best Demo Prize at ICCV 2015. Original Caffe-based code of this project can be found here. Results produced with this Keras/Tensorflow code are almost identical to that with the Caffe-based version.

If you use this code/model for your research, please cite the following paper:

@inproceedings{crfasrnn_ICCV2015,
    author = {Shuai Zheng and Sadeep Jayasumana and Bernardino Romera-Paredes and Vibhav Vineet and
    Zhizhong Su and Dalong Du and Chang Huang and Philip H. S. Torr},
    title  = {Conditional Random Fields as Recurrent Neural Networks},
    booktitle = {International Conference on Computer Vision (ICCV)},
    year   = {2015}
}

Installation Guide

Step 1: Clone the repository

$ git clone https://github.com/sadeepj/crfasrnn_keras.git

The root directory of the clone will be referred to as crfasrnn_keras hereafter.

Step 2: Install dependencies

Note: If you are using a Python virtualenv, make sure it is activated before running each command in this guide.

Use the requirements.txt file (or requirements_gpu.txt, if you have a GPU device) in this repository to install all the dependencies via pip:

$ cd crfasrnn_keras
$ pip install -r requirements.txt  # If you have a GPU device, use requirements_gpu.txt instead

As you can notice from the contents of requirements.txt, we only depend on tensorflow, keras, and h5py. Additionally, Pillow is required for running the demo. After installing the dependencies, run the following commands to make sure they are properly installed:

$ python
>>> import tensorflow
>>> import keras

You should not see any errors while importing tensorflow and keras above.

Step 3: Build CRF-RNN custom op C++ code

Run make inside the crfasrnn_keras/src/cpp directory:

$ cd crfasrnn_keras/src/cpp
$ make

Note that the python command in the console should refer to the Python interpreter associated with your Tensorflow installation before running the make command above.

You will get a new file named high_dim_filter.so from this build. If it fails, refer to the official Tensorflow guide for building a custom op for help.

Note: This make script works on Linux and macOS, but not on Windows OS. If you are on Windows, please check this issue and the comments therein for build instructions. The official Tensorflow guide for building a custom op does not yet include build instructions for Windows.

Step 4: Download the pre-trained model weights

Download the model weights from here or here and place it in the crfasrnn_keras directory with the file name crfrnn_keras_model.h5.

Step 5: Run the demo

$ cd crfasrnn_keras
$ python run_demo.py

If all goes well, you will see the segmentation results in a file named "labels.png".

Limitations of the Current Version

1. Currently, some operations in the CRF-RNN layer can only run on the CPU. An all-GPU version will be released soon.
2. Current implementation of CrfRnnLayer only supports batch_size == 1