SpA-Former:An Effective and Lightweight Transformer for Image Shadow Removal(IJCNN2023 Oral)

new

• 2023.8.4 IEEE EXPORE have published here:[https://ieeexplore.ieee.org/document/10191081}

• 2023.4.7 The paper is accepted in IJCNN 2023! Thanks for your help

• 2022.6.30 The draft is released now at http://arxiv.org/abs/2206.10910 SpA-Former:Transformer image shadow detection and removal via spatial attention.

• You can play it on Google colab Thanks for the author.

• There is SpA-Former Demo testing guidelines

Attention map of SpA-Former on ISTD dataset

Qucikly run

1. TRAIN

Modify the config.yml to set your parameters and run:

python train.py

2. TEST

First，the dataset is trained on 640x480, so you should resize test dataset to 640X480, you can use the code to resize your image bash python bigresize.py and then follow the code to test the results:

python predict.py --config <path_to_config.yml_in_the_out_dir> --test_dir <path_to_a_directory_stored_test_data> --out_dir <path_to_an_output_directory> --pretrained <path_to_a_pretrained_model> --cuda

Attention visual results is bellow:Attention visual results.

There're my pre-trained models on ISTD(./pretrained_models/ISTD/gen_model_epoch_200.pth)

3. Pretrained model

Download the pretrained model shadow-removal Google-drive and Baidu Drive 提取码：rpis

4.Test results

Our test results: Google-drive and Baidu drive 提取码：18ut

5.Evaluate

To reproduce PSNR/SSIM/RMSE scores of the paper, run MATLAB script

evaluate.m

In this section, I compares SpA-Former with several methods using peak signal to noise ratio (PSNR) and structural similarity index (SSIM) and (RMSE) as metrics on datasets ISTD.

)

ACKNOLAGEMENT

The code is updated on [https://github.com/Penn000/SpA-GAN_for_cloud_removal)]

2. DATASET

2.1. ISTD_DATASET

Click official address Build the file structure as the folder data shown. Here input is the folder where the shadow image is stored and the folder target stores the corresponding no shadow images.

./
+-- data
    +--	ISTD_DATASET
        +-- train
        |   +-- input
        |   |   +-- 0.png
        |   |   +-- ...
        |   +-- target
        |       +-- 0.png
        |       +-- ...
        +-- test
            +-- input
            |   +-- 0.png
            |   +-- ...
            +-- target
                +-- 0.png
                +-- ...

CONTACT

Contact me if you have any questions about the code and its execution.

E-mail: framebreak@sjtu.edu.cn

If you think this work is helpful for your research, give me a star :-D

Citations

@INPROCEEDINGS{10191081,
  author={Zhang, Xiaofeng and Zhao, Yudi and Gu, Chaochen and Lu, Changsheng and Zhu, Shanying},
  booktitle={2023 International Joint Conference on Neural Networks (IJCNN)}, 
  title={SpA-Former:An Effective and lightweight Transformer for image shadow removal}, 
  year={2023},
  volume={},
  number={},
  pages={1-8},
  doi={10.1109/IJCNN54540.2023.10191081}}