Skip to content

Semantic Segmentation for Aerial / Satellite Images with Convolutional Neural Networks including an unofficial implementation of Volodymyr Mnih's methods

License

Notifications You must be signed in to change notification settings

mitmul/ssai-cnn

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

80 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

This is an implementation of Volodymyr Mnih's dissertation methods on his Massachusetts road & building dataset and my original methods that are published in this paper.

Requirements

  • Python 3.5 (anaconda with python 3.5.1 is recommended)
    • Chainer 1.5.0.2
    • Cython 0.23.4
    • NumPy 1.10.1
    • tqdm
  • OpenCV 3.0.0
  • lmdb 0.87
  • Boost 1.59.0
  • Boost.NumPy (26aaa5b)

Build Libraries

OpenCV 3.0.0

$ wget https://github.com/Itseez/opencv/archive/3.0.0.zip
$ unzip 3.0.0.zip && rm -rf 3.0.0.zip
$ cd opencv-3.0.0 && mkdir build && cd build
$ bash $SSAI_HOME/shells/build_opencv.sh
$ make -j32 install

If some libraries are missing, do below before compiling 3.0.0.

$ sudo apt-get install -y libopencv-dev libtbb-dev

Boost 1.59. 0

$ wget http://downloads.sourceforge.net/project/boost/boost/1.59.0/boost_1_59_0.tar.bz2
$ tar xvf boost_1_59_0.tar.bz2 && rm -rf boost_1_59_0.tar.bz2
$ cd boost_1_59_0
$ ./bootstrap.sh
$ ./b2 -j32 install cxxflags="-I/home/ubuntu/anaconda3/include/python3.5m"

Boost.NumPy

$ git clone https://github.com/ndarray/Boost.NumPy.git
$ cd Boost.NumPy && mkdir build && cd build
$ cmake -DPYTHON_LIBRARY=$HOME/anaconda3/lib/libpython3.5m.so ../
$ make install

Build utils

$ cd $SSAI_HOME/scripts/utils
$ bash build.sh

Create Dataset

$ bash shells/download.sh
$ bash shells/create_dataset.sh
Dataset Training Validation Test
mass_roads 8580352 108416 379456
mass_roads_mini 1060928 30976 77440
mass_buildings 1060928 30976 77440
mass_merged 1060928 30976 77440

Start Training

$ CHAINER_TYPE_CHECK=0 CHAINER_SEED=$1 \
nohup python scripts/train.py \
--seed 0 \
--gpu 0 \
--model models/MnihCNN_multi.py \
--train_ortho_db data/mass_merged/lmdb/train_sat \
--train_label_db data/mass_merged/lmdb/train_map \
--valid_ortho_db data/mass_merged/lmdb/valid_sat \
--valid_label_db data/mass_merged/lmdb/valid_map \
--dataset_size 1.0 \
> mnih_multi.log 2>&1 < /dev/null &

Prediction

python scripts/predict.py \
--model results/MnihCNN_multi_2016-02-03_03-34-58/MnihCNN_multi.py \
--param results/MnihCNN_multi_2016-02-03_03-34-58/epoch-400.model \
--test_sat_dir data/mass_merged/test/sat \
--channels 3 \
--offset 8 \
--gpu 0 &

Evaluation

$ PYTHONPATH=".":$PYTHONPATH python scripts/evaluate.py \
--map_dir data/mass_merged/test/map \
--result_dir results/MnihCNN_multi_2016-02-03_03-34-58/ma_prediction_400 \
--channel 3 \
--offset 8 \
--relax 3 \
--steps 1024

Results

Conventional methods

Model Mass. Buildings Mass. Roads Mass.Roads-Mini
MnihCNN 0.9150 0.8873 N/A
MnihCNN + CRF 0.9211 0.8904 N/A
MnihCNN + Post-processing net 0.9203 0.9006 N/A
Single-channel 0.9503062 0.91730195 (epoch 120) 0.89989258
Single-channel with MA 0.953766 0.91903522 (epoch 120) 0.902895

Multi-channel models (epoch = 400, step = 1024)

Model Building-channel Road-channel Road-channel (fixed)
Multi-channel 0.94346856 0.89379946 0.9033020025
Multi-channel with MA 0.95231262 0.89971473 0.90982972
Multi-channel with CIS 0.94417078 0.89415726 0.9039476538
Multi-channel with CIS + MA 0.95280431 0.90071099 0.91108087

Test on urban areas (epoch = 400, step = 1024)

Model Building-channel Road-channel
Single-channel with MA 0.962133 0.944748
Multi-channel with MA 0.962797 0.947224
Multi-channel with CIS + MA 0.964499 0.950465

x0_sigma for inverting feature maps

159.348674296

After prediction for single MA

$ bash shells/predict.sh
$ python scripts/integrate.py --result_dir results --epoch 200 --size 7,60
$ PYTHONPATH=".":$PYTHONPATH python scripts/evaluate.py --map_dir data/mass_merged/test/map --result_dir results/integrated_200 --channel 3 --offset 8 --relax 3 --steps 256
$ PYTHONPATH="." python scripts/eval_urban.py --result_dir results/integrated_200 --test_map_dir data/mass_merged/test/map --steps 256

Pre-trained models and Predicted results

Reference

If you use this code for your project, please cite this journal paper:

Shunta Saito, Takayoshi Yamashita, Yoshimitsu Aoki, "Multiple Object Extraction from Aerial Imagery with Convolutional Neural Networks", Journal of Imaging Science and Technology, Vol. 60, No. 1, pp. 10402-1-10402-9, 2015

About

Semantic Segmentation for Aerial / Satellite Images with Convolutional Neural Networks including an unofficial implementation of Volodymyr Mnih's methods

Resources

License

Stars

Watchers

Forks

Packages

No packages published