Official code of "Birds of A Feather Flock Together"

Birds of A Feather Flock Together: Category-Divergence Guidance for Domain Adaptive Segmentation. IEEE Transactions on Image Processing, 2022.

Bo Yuan^1,2, Danpei Zhao^1,2*, Shuai Shao³, Zehuan Yuan³, Changhu Wang³.

¹ _{Image Processing Center, BUAA}
² _{Airvic Lab}
³ _{ByteDance AI-Lab}

Paper link

Abstract

Unsupervised domain adaptation (UDA) aims to enhance the generalization capability of a certain model from a source domain to a target domain. Present UDA models focus on alleviating the domain shift by minimizing the feature discrepancy between the source domain and the target domain but usually ignore the class confusion problem. In this work, we propose an Inter-class Separation and Intra-class Aggregation (ISIA) mechanism. It encourages the cross-domain representative consistency between the same categories and differentiation among diverse categories. In this way, the features belonging to the same categories are aligned together and the confusable categories are separated. By measuring the align complexity of each category, we design an Adaptive-weighted Instance Matching (AIM) strategy to further optimize the instance-level adaptation. Based on our proposed methods, we also raise a hierarchical unsupervised domain adaptation framework for cross-domain semantic segmentation task. Through performing the image-level, feature-level, category-level and instance-level alignment, our method achieves a stronger generalization performance of the model from the source domain to the target domain. In two typical cross-domain semantic segmentation tasks, i.e., GTA5 to Cityscapes and SYNTHIA to Cityscapes, our method achieves the state-of-the-art segmentation accuracy. We also build two cross-domain semantic segmentation datasets based on the publicly available data, i.e., remote sensing building segmentation and road segmentation, for domain adaptive segmentation.

Citation

@ARTICLE{9745831,  
  author={Yuan, Bo and Zhao, Danpei and Shao, Shuai and Yuan, Zehuan and Wang, Changhu},  
  journal={IEEE Transactions on Image Processing},   
  title={Birds of a Feather Flock Together: Category-Divergence Guidance for Domain Adaptive Segmentation},   
  year={2022},  
  volume={31},  
  number={},  
  pages={2878-2892},  
  doi={10.1109/TIP.2022.3162471}
}

Update 2022-03-20

Preparation

git clone https://github.com/HibiscusYB/BAFFT.git     
cd BAFFT

install inplace-abn

pip install inplace-abn  

install apex

git clone https://github.com/NVIDIA/apex 
cd apex/      
pip setup.py install   

Prepare Dataset

Download Cityscapes, CycleGAN transferred GTA5 and gta5 labels. Symbolic link them under data folder:

ln -s path_to_Cityscapes_folder ./data/Cityscapes   
ln -s path_to_gta5_deeplab_folder ./data/gta5_deeplab    
ln -s path_to_gta5_labels_folder ./data/gta5_deeplab/labels       

pretrained models

Download models and put them in ./pretrained folder

BaiduNetDesk: link: https://pan.baidu.com/s/1n_l9kJNFda5vNTKDE7b92g
fetch code: j5a3

Google Drive: https://drive.google.com/drive/u/0/folders/1pMf0nQ5eawmGHlSQ9FWmHgboROtCfbFk

Quick Start

GTA5 to Cityscapes:

Train:

Step 1

CUDA_VISIBLE_DEVICES=0 python train_step1.py  
python Generate_Pseudo_Label.py   

Step 2

CUDA_VISIBLE_DEVICES=0 python train_step2.py  

Inference & Evaluation

python evaluate_cityscapes.py --restore-from ./pretrained/GTA2Cityscapes/GTA5_Best.pth   
python compute_iou.py ./data/Cityscapes/gtFine/val result/cityscapes   

Results

GTA5 to Cityscapes

SYNTHIA to Cityscapes

Visualizations

License

All Rights Reserved