nighttime_dehaze (ACMMM'2023)

Introduction

Enhancing Visibility in Nighttime Haze Images Using Guided APSF and Gradient Adaptive Convolution
ACM International Conference on Multimedia (ACMMM2023)
Yeying Jin*, Beibei Lin*, Wending Yan, Yuan Yuan, Wei Ye, and Robby T. Tan

Prerequisites

git clone https://github.com/jinyeying/nighttime_dehaze.git
cd nighttime_dehaze/
conda create -n dehaze python=3.7
conda activate dehaze
conda install pytorch=1.10.2 torchvision torchaudio cudatoolkit=11.3 -c pytorch
python3 -m pip install scipy==1.7.3
python3 -m pip install opencv-python==4.4.0.46

Nighttime Haze Data

Data	Dropbox	BaiduPan	Number & Type
RealNightHaze	Dropbox	BaiduPan code:r5mi	443, Haze
Internet_night_clean1	Dropbox	BaiduPan code:m7k1	411, Clean Reference
Internet_night_clean2	Dropbox	BaiduPan code:8f13	50, Clean Reference
GTA5 nighttime fog	Dropbox	BaiduPan code:67ml	Train:787,Test:77, Synthetic

Synthetic Nighttime Haze and Clean Ground Truth

• ECCV2020 Nighttime Defogging Using High-Low Frequency Decomposition and Grayscale-Color Networks [Paper]
  Wending Yan, Robby T. Tan and Dengxin Dai

Nighttime Dehazing Results Dropbox | BaiduPan code:oovt

Model	Dropbox	BaiduPan	Model Put in Path	Results Dropbox	Results BaiduPan
dehaze.pt	dehaze.pt	dehaze.pt code:n3t8	results/dehaze/model	RealNightHaze	RealNightHaze code:i43f
GTA5.pt	GTA5.pt	GTA5.pt code:fk29	results/GTA5/model	GTA5	GTA5 code:ufen
NHR.pt	NHR.pt	NHR.pt code:dnhf	results/NHR/model	NHR	NHR code:0nma
NHM.pt	NHM.pt	NHM.pt code:d7oj	results/NHM/model	NHM	NHM code：4gt0
NHC.pt	NHC.pt	NHC.pt code:yryp	results/NHC/model	NHC	NHC code：njf9

We provide the visualization results in 0_ACMMM23_RESULTS/NHR/index.html,
inside the directory 0_ACMMM23_RESULTS/NHR/img_0/ are hazy inputs, img_1 are ground truths, img_2 are our results.
For results corresponding to GTA5, NHM or NHC, please refer to the respective directories.

• For the RealNightHaze Dataset

1. Set the datasetpath to RealNightHaze,
2. Download the checkpoint dehaze.pt, put in results/dehaze/model,
3. Run the Python code, results are in results/dehaze/output.

CUDA_VISIBLE_DEVICES=1 python main_test.py --dataset dehaze --datasetpath /diskc/yeying/night_dehaze/dataset/Internet_night_fog/

• For the Synthetic Dataset

1. Set Line18 --have_gt to True, set the datasetpath to GTA5 or NHR or NHM or NHC,
2. Download the checkpoint GTA5.pt, put in results/GTA5/model. Similarly, for NHR.pt, NHM.pt, NHC.pt,
3. Run the Python code,

CUDA_VISIBLE_DEVICES=1 python main_test.py --dataset NHM --datasetpath /diskc/yeying/night_dehaze/dataset/middlebury/testA/ 
CUDA_VISIBLE_DEVICES=1 python main_test.py --dataset NHC --datasetpath /diskc/yeying/night_dehaze/dataset/Cityscape/testA/ 
CUDA_VISIBLE_DEVICES=1 python main_test.py --dataset NHR --datasetpath /diskc/yeying/night_dehaze/dataset/NHR/testA/ 
CUDA_VISIBLE_DEVICES=1 python main_test.py --dataset GTA5 --datasetpath /diskc/yeying/night_dehaze/GTA5/testA/

• Evaluation: Set the dataset_name GTA5 or NHR or NHM or NHC, and run the Python code:

python calculate_psnr_ssim_NH_GTA5.py

Dataset	PSNR	SSIM
GTA5	30.383	0.9042
NHR	26.56	0.89
NHM	33.76	0.92
NHC	38.86	0.97

APSF-Guided Nighttime Glow Rendering

Run the Matlab code to obtain the clean and glow pairs:

APSF_GLOW_RENDER_CODE/synthetic_glow_pairs.m

Change the datapath nighttime_dehaze/paired_data/clean_data/,
the paired clean and glow results are saved in nighttime_dehaze/paired_data/clean/ and nighttime_dehaze/paired_data/glow/,
the visualization of light source maps are in nighttime_dehaze/paired_data/glow_render_visual/light_source/.

Run the Matlab code to visualize the Fig.3 in the main paper:

APSF_GLOW_RENDER_CODE/synthetic_glow_fig3_visualization.m

APSF and Alpha Matting are the implementations of the papers:

• CVPR03 Shedding Light on the Weather [Paper]
• CVPR06 A Closed-Form Solution to Natural Image Matting [Paper]

License

The code and models in this repository are licensed under the MIT License for academic and other non-commercial uses.
For commercial use of the code and models, separate commercial licensing is available. Please contact:

• Yeying Jin (jinyeying@u.nus.edu)
• Robby T. Tan (tanrobby@gmail.com)
• Jonathan Tan (jonathan_tano@nus.edu.sg)

Citation

If this work or the Internet data is useful for your research, please cite our paper.

@inproceedings{jin2023enhancing,
  title={Enhancing visibility in nighttime haze images using guided apsf and gradient adaptive convolution},
  author={Jin, Yeying and Lin, Beibei and Yan, Wending and Yuan, Yuan and Ye, Wei and Tan, Robby T},
  booktitle={Proceedings of the 31st ACM International Conference on Multimedia},
  pages={2446--2457},
  year={2023}
}

@inproceedings{jin2022unsupervised,
  title={Unsupervised night image enhancement: When layer decomposition meets light-effects suppression},
  author={Jin, Yeying and Yang, Wenhan and Tan, Robby T},
  booktitle={European Conference on Computer Vision},
  pages={404--421},
  year={2022},
  organization={Springer}
}