By Lin Song, Yanwei Li, Zeming Li, Gang Yu, Hongbin Sun, Jian Sun, Nanning Zheng.
This project provides a cuda implementation for "Learnable Tree Filter for Structure-preserving Feature Transform" (NeurIPS2019) on PyTorch. Multiple semantic segmentation experiments are reproduced to verify the effectiveness of tree filtering module on PASCAL VOC2012 and Cityscapes. For the reason that the experiments in the paper were conducted using internal framework, this project reimplements them on PyTorch and reports detailed comparisons below. In addition, many thanks to TorchSeg.
- PyTorch 1.2
sudo pip3 install torch torchvision
- Easydict
sudo pip3 install easydict
- Apex
https://nvidia.github.io/apex/index.html
- Ninja
sudo apt-get install ninja-build
- tqdm
sudo pip3 install tqdm
- Boost (optional for Prim and Kruskal algorithm)
sudo apt-get install libboost-dev
git clone https://github.com/StevenGrove/TreeFilter-Torchcd TreeFilter-Torch/furnace/kernels/lib_tree_filtersudo python3 setup.py build develop
This project implements three well-known algorithms of minimal spanning tree, i.e., Boruvka, Kruskal and Prim. The default algorithm is set to Boruvka for its linear computational complexity in the plain graph. The user can change the configuration in the source file "lib_tree_filter/src/mst/mst.cu" .
- ResNet-50 GoogleDrive
- ResNet-101 GoogleDrive
FCN-32d: FCN with decoder whose maximum stride is 32;
Extra: Global average pooling + ResBlock;
TF: Learnable tree filtering module;
SS: Single-scale;
MSF: Multi-scale + Flip.
| Methods | Backbone | mIoU (ss) | Acc (ss) | mIoU (msf) | Acc (msf) | Model |
|---|---|---|---|---|---|---|
| FCN-32d | R50_v1c | 71.82% | 93.62% | 73.96% | 94.14% | GoogleDrive |
| FCN-32d+TF | R50_v1c | 76.31% | 94.57% | 77.80% | 94.96% | GoogleDrive |
| FCN-32d | R101_v1c | 74.53% | 94.29% | 76.08% | 94.63% | GoogleDrive |
| FCN-32d+TF | R101_v1c | 77.82% | 94.92% | 79.22% | 95.22% | GoogleDrive |
| FCN-32d+Extra | R101_v1c | 78.04% | 95.01% | 79.69% | 95.41% | GoogleDrive |
| FCN-32d+Extra+TF | R101_v1c | 79.81% | 95.38% | 80.97% | 95.67% | GoogleDrive |
| FCN-32d+Extra+TF* | R101_v1c | 80.32% | 95.66% | 82.28% | 96.01% | GoogleDrive |
* further finetuned on the original train set
| Methods | Backbone | mIoU (ss) | Acc (ss) | mIoU (msf) | Acc (msf) | Model |
|---|---|---|---|---|---|---|
| FCN-32d+Extra | R101_v1c | 78.29% | 96.09% | 79.40% | 96.27% | GoogleDrive |
| FCN-32d+Extra+TF | R101_v1c | 79.58% | 96.31% | 80.85% | 96.46% | GoogleDrive |
As in the original TorchSeg, distributed training is recommended for either single machine or multiple machines.
For detailed usage, please refer to the Training and Inference sections in TorchSeg.
- Experiments on ADE20K
- Visualization of tree filter
- Additional tasks
- Object detection
- Instance segmentation
- Optical flow
Please cite the learnable tree filter in your publications if it helps your research.
@inproceedings{song2019learnable,
title = {Learnable Tree Filter for Structure-preserving Feature Transform},
author = {Song, Lin and Li, Yanwei and Li, Zeming and Yu, Gang and Sun, Hongbin and Sun, Jian and Zheng, Nanning},
booktitle = {Advances in Neural Information Processing Systems},
year = {2019}
}
Please cite this project in your publications if it helps your research.
@misc{treefilter-torch,
author = {Song, Lin},
title = {TreeFiler-Torch},
howpublished = {\url{https://github.com/StevenGrove/TreeFilter-Torch}},
year ={2019}
}