NTRENet:Learning Non-target Knowledge for Few-shot Semantic Segmentation

This repo contains the code for our CVPR 2022 paper "Learning Non-target Knowledge for Few-shot Semantic Segmentation" by Yuanwei Liu, Nian Liu, Qinglong Cao, Xiwen Yao, Junwei Han, Ling Shao.

Abstract: Existing studies in few-shot semantic segmentation only focus on mining the target object information, however, often are hard to tell ambiguous regions, especially in non-target regions, which include background (BG) and Distracting Objects (DOs). To alleviate this problem, we propose a novel framework, namely Non-Target Region Eliminating (NTRE) network, to explicitly mine and eliminate BG and DO regions in the query. First, a BG Mining Module (BGMM) is proposed to extract the BG region via learning a general BG prototype. To this end, we design a BG loss to supervise the learning of BGMM only using the known target object segmentation ground truth. Then, a BG Eliminating Module and a DO Eliminating Module are proposed to successively filter out the BG and DO information from the query feature, based on which we can obtain a BG and DO-free target object segmentation result. Furthermore, we propose a prototypical contrastive learning algorithm to improve the model ability of distinguishing the target object from DOs. Extensive experiments on both PASCAL- 5ⁱ and COCO- 20ⁱ datasets show that our approach is effective despite its simplicity.

Dependencies

• Python 3.8
• PyTorch 1.7.0
• cuda 11.0
• torchvision 0.8.1
• tensorboardX 2.14

Datasets

• PASCAL-5ⁱ: VOC2012 + SBD

• COCO-20ⁱ: COCO2014

References

This repo is mainly built based on PFENet. Thanks for their great work!

BibTeX

If you find our work and this repository useful. Please consider giving a star ⭐ and citation 📚.

@InProceedings{Liu_2022_CVPR,
    author    = {Liu, Yuanwei and Liu, Nian and Cao, Qinglong and Yao, Xiwen and Han, Junwei and Shao, Ling},
    title     = {Learning Non-Target Knowledge for Few-Shot Semantic Segmentation},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {11573-11582}
}