Awesome Diffusion Models In Low-level Vision

🔥A curated list of awesome Diffusion Models(DMs) in low-level vision.🔥

Please feel free to offer your suggestions in the Issues and pull requests to add links.

[ Last updated at 2024/05/21 ]

Latest Works Recommended
Papers
- Diffusion Models For Natural Image Restoration
  General-purpose Image Restoration (IR)
  - Zero-shot DM-based IR
  - Supervised DM-based IR
  Task-specific Image Restoration (IR)
- Extended Diffusion Models In Low-level Vision
  Diffusion Models In Low-level Medical Image Analysis
  Diffusion Models In Remote Sensing For Low-level Vision Tasks
  Varied Low-level Vision Tasks In Video Through Diffusion Models

Related Surveys Recommended

Datasets for low-level vision tasks

Reference

Latest Works Recommended

Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model
Chunming He, Chengyu Fang, Yulun Zhang, Kai Li, Longxiang Tang, Chenyu You, Fengyang Xiao, Zhenhua Guo, Xiu Li
arXiv 2023. [Paper] [Code]
Nov. 2023

Papers

1. Diffusion Models For Natural Image Restoration

1.1 General-purpose Image Restoration

1.1.1 Zero-shot DM-based IR

Release	Method	Title	Pub.	Link
2023/05	DiffPIR	Denoising diffusion models for plug-and-play image restoration ^{_{Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, Luc Van Gool}}	CVPR 2023	Paper/Code
2023/05	RED-Diff	A Variational Perspective on Solving Inverse Problems with Diffusion Models ^{_{Morteza Mardani, Jiaming Song, Jan Kautz, Arash Vahdat}}	arXiv 2023	Paper/
2023/04	GDP	Generative Diffusion Prior for Unified Image Restoration and Enhancement ^{_{Ben Fei, Zhaoyang Lyu, Liang Pan, Junzhe Zhang, Weidong Yang, Tianyue Luo, Bo Zhang, Bo Dai}}	CVPR 2023	Paper/Code
2023/04	-	Score-Based Diffusion Models as Principled Priors for Inverse Imaging ^{_{Berthy T. Feng, Jamie Smith, Michael Rubinstein, Huiwen Chang, Katherine L. Bouman, William T. Freeman}}	arXiv 2023	Paper/
2023/02	πGDM	Pseudoinverse-Guided Diffusion Models for Inverse Problems ^{_{Jiaming Song, Arash Vahdat, Morteza Mardani, Jan Kautz}}	ICLR 2023	Paper/
2022/12	ADIR	ADIR: Adaptive Diffusion for Image Reconstruction ^{_{Shady Abu-Hussein, Tom Tirer, Raja Giryes}}	arXiv 2022	Paper/
2022/12	DDNM	Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model ^{_{Yinhuai Wang, Jiwen Yu, Jian Zhang}}	ICLR 2023	Paper/Code
2022/09	DPS	Diffusion Posterior Sampling for General Noisy Inverse Problems ^{_{Hyungjin Chung, Jeongsol Kim, Michael T. Mccann, Marc L. Klasky, Jong Chul Ye}}	CVPR 2023	Paper/Code
2022/01	MCG	Improving diffusion models for inverse problems using manifold constraints ^{_{Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, Jong Chul Ye}}	NIPS 2022	Paper/Code
2022/01	DDRM	Denoising Diffusion Restoration Models ^{_{Bahjat Kawar, Michael Elad, Stefano Ermon, Jiaming Song}}	NIPS 2022	Paper/Code
2021/12	CCDF	Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction ^{_{Hyungjin Chung, Byeongsu Sim, Jong Chul Ye}}	CVPR 2022	Paper/
2021/08	ILVR	ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models ^{_{Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, Sungroh Yoon}}	ICCV 2021	Paper/Code
2021/05	SNIPS	SNIPS: Solving Noisy Inverse Problems Stochastically ^{_{Bahjat Kawar, Gregory Vaksman, Michael Elad}}	NIPS 2021	Paper/Code

1.1.2 Supervised DM-based IR

Release	Method	Title	Pub.	Link
2024/03	DiffUIR	Selective Hourglass Mapping for Universal Image Restoration Based on Diffusion Model ^{_{Dian Zheng, Xiao-Ming Wu, Shuzhou Yang, Jian Zhang, Jian-Fang Hu, Wei-Shi Zheng}}	CVPR 2024	Paper/Code
2024/03	-	Efficient Diffusion Model for Image Restoration by Residual Shifting ^{_{Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, Luc Van Gool}}	arXiv 2024	Paper/Code
2023/08	RDDM	Residual Denoising Diffusion Models ^{_{Jiawei Liu, Qiang Wang, Huijie Fan, Yinong Wang, Yandong Tang, Liangqiong Qu}}	CVPR 2024	Paper/Code
2023/08	DiffBIR	DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior ^{_{Xinqi Lin, Jingwen He, Ziyan Chen, Zhaoyang Lyu, Bo Dai, Fanghua Yu, Wanli Ouyang, Yu Qiao, Chao Dong}}	arXiv 2023	Paper/Code
2023/05	InDI	Inversion by Direct Iteration: An Alternative to Denoising Diffusion for Image Restoration ^{_{Mauricio Delbracio, Peyman Milanfar}}	arXiv 2023	Paper/Code
2023/03	DiracDiffusion	DiracDiffusion: Denoising and Incremental Reconstruction with Assured Data-Consistency ^{_{Zalan Fabian, Berk Tinaz, Mahdi Soltanolkotabi}}	arXiv 2023	Paper/
2023/04	Refusion	Refusion: Enabling Large-Size Realistic Image Restoration with Latent-Space Diffusion Models ^{_{Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao, Jens Sjölund, Thomas B. Schön}}	CVPRW 2023	Paper/Code
2023/01	IR-SDE	Image Restoration with Mean-Reverting Stochastic Differential Equations ^{_{Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao, Jens Sjölund, Thomas B. Schön}}	ICML 2023	Paper/Code
2021/12	LDM	High-resolution image synthesis with latent diffusion models ^{_{Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, Björn Ommer}}	CVPR 2022	Paper/Code

1.2 Task-specific Image Restoration

1.2.1 Super Resolution (SR)

Release	Method	Title	Pub.	Link
2023/11	CoSeR	CoSeR: Bridging Image and Language for Cognitive Super-Resolution ^{_{Haoze Sun, Wenbo Li, Jianzhuang Liu, Haoyu Chen, Renjing Pei, Xueyi Zou, Youliang Yan, Yujiu Yang}}	arXiv 2023	Paper/Code
2023/09	-	License Plate Super-Resolution Using Diffusion Models ^{_{Sawsan AlHalawani, Bilel Benjdira, Adel Ammar, Anis Koubaa, Anas M. Ali}}	arXiv 2023	Paper/
2023/08	PASD	Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization ^{_{Tao Yang, Rongyuan Wu, Peiran Ren, Xuansong Xie, Lei Zhang}}	arXiv 2023	Paper/Code
2023/07	ResShift	ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting ^{_{Zongsheng Yue, Jianyi Wang, Chen Change Loy}}	NIPS 2023	Paper/Code
2023/07	PartDiff	PartDiff: Image Super-resolution with Partial Diffusion Models ^{_{Axi Niu, Pham Xuan Trung, Kang Zhang, Jinqiu Sun, Yu Zhu, In So Kweon, Yanning Zhang}}	arXiv 2023	Paper/
2023/07	ACDMSR	ACDMSR: Accelerated Conditional Diffusion Models for Single Image Super-Resolution ^{_{Axi Niu, Pham Xuan Trung, Kang Zhang, Jinqiu Sun, Yu Zhu, In So Kweon, Yanning Zhang}}	IEEE Trans. Broadcast. 2024	Paper/
2023/05	StableSR	Exploiting Diffusion Prior for Real-World Image Super-Resolution ^{_{Jianyi Wang, Zongsheng Yue, Shangchen Zhou, Kelvin C.K. Chan, Chen Change Loy}}	arXiv 2023	Paper/Code
2023/03	DR2	DR2: Diffusion-Based Robust Degradation Remover for Blind Face Restoration ^{_{Zhixin Wang, Xiaoyun Zhang, Ziying Zhang, Huangjie Zheng, Mingyuan Zhou, Ya Zhang, Yanfeng Wang}}	CVPR 2023	Paper/Code
2023/03	ResDiff	ResDiff: Combining CNN and Diffusion Model for Image Super-Resolution ^{_{Shuyao Shang, Zhengyang Shan, Guangxing Liu, Jinglin Zhang}}	AAAI 2024	Paper/Code
2023/03	IDM	Implicit Diffusion Models for Continuous Super-Resolution ^{_{Sicheng Gao, Xuhui Liu, Bohan Zeng, Sheng Xu, Yanjing Li, Xiaoyan Luo, Jianzhuang Liu, Xiantong Zhen, Baochang Zhang}}	CVPR 2023	Paper/Code
2023/02	-	Denoising Diffusion Probabilistic Models for Robust Image Super-Resolution in the Wild ^{_{Hshmat Sahak, Daniel Watson, Chitwan Saharia, David Fleet}}	arXiv 2023	Paper/
2023/02	CDPMSR	CDPMSR: Conditional Diffusion Probabilistic Models for Single Image Super-Resolution ^{_{Axi Niu, Kang Zhang, Trung X. Pham, Jinqiu Sun, Yu Zhu, In So Kweon, Yanning Zhang}}	arXiv 2023	Paper/
2022/09	SUE-SR	Face Super-Resolution Using Stochastic Differential Equations ^{_{Marcelo dos Santos, Rayson Laroca, Rafael O. Ribeiro, João Neves, Hugo Proença, David Menotti}}	SIGGRAPH 2022	Paper/Code
2021/05	CDM	Cascaded Diffusion Models for High Fidelity Image Generation ^{_{Jonathan Ho, Chitwan Saharia, William Chan, David J. Fleet, Mohammad Norouzi, Tim Salimans.}}	JMLR 2022	Paper/Code
2021/04	SR3	Image Super-Resolution via Iterative Refinement ^{_{Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J. Fleet, Mohammad Norouzi.}}	TPAMI 2022	Paper/Code
2021/04	SRDiff	SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models ^{_{H. Li, Y. Yang, M. Chang, S. Chen, H. Feng, Z. Xu, Q. Li, and Y. Chen.}}	Neurocomputing 2022	Paper/Code

1.2.2 Inpainting

Release	Method	Title	Pub.	Link
2024/03	MMGInpainting	MMGInpainting: Multi-Modality Guided Image Inpainting Based On Diffusion Models ^{_{Cong Zhang, Wenxia Yang, Xin Li, Huan Han}}	IEEE Trans Multimedia 2024	Paper/Code
2024/03	BrushNet	BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion ^{_{Xuan Ju, Xian Liu, Xintao Wang, Yuxuan Bian, Ying Shan, Qiang Xu}}	arXiv 2024	Paper/Code
2024/03	-	Fill in the ____ (a Diffusion-based Image Inpainting Pipeline) ^{_{Eyoel Gebre, Krishna Saxena, Timothy Tran}}	arXiv 2024	Paper/
2023/09	Gradpaint	Gradpaint: Gradient-Guided Inpainting with Diffusion Models ^{_{Asya Grechka, Guillaume Couairon, Matthieu Cord}}	CVIU 2024	Paper/
2022/05	Palette	Palette: Image-to-image diffusion models ^{_{C. Saharia, W. Chan, H. Chang, C. Lee, J. Ho, T. Salimans, D. Fleet, and M. Norouzi.}}	SIGGRAPH 2022	Paper/Code
2022/01	RePaint	Repaint: Inpainting using denoising diffusion probabilistic models ^{_{A. Lugmayr, M. Danelljan, A. Romero, F. Yu, R. Timofte, and L. Van Gool.}}	CVPR 2022	Paper/Code

1.2.3 Deblur

Release	Method	Title	Pub.	Link
2024/04	Diffevent	Diffevent: Event Residual Diffusion for Image Deblurring ^{_{Pei Wang, Jiumei He, Qingsen Yan, Yu Zhu, Jinqiu Sun, Yanning Zhang}}	ICASSP 2024	Paper/Code
2024/01	FastDiffusionEM	Fast Diffusion EM: a diffusion model for blind inverse problems with application to deconvolution ^{_{Charles Laroche, Andrés Almansa, Eva Coupete}}	WACV 2024	Paper/Code
2024/01	SI-DDPM-FMO	Single-Image Deblurring, Trajectory and Shape Recovery of Fast Moving Objects With Denoising Diffusion Probabilistic Models ^{_{Radim Spetlik, Denys Rozumnyi, Jiří Matas}}	WACV 2024	Paper/Code
2023/05	HI-Diff	Hierarchical Integration Diffusion Model for Realistic Image Deblurring ^{_{Zheng Chen, Yulun Zhang, Ding Liu, Bin Xia, Jinjin Gu, Linghe Kong, Xin Yuan}}	arXiv 2023	Paper/
2022/12	-	Multiscale Structure Guided Diffusion for Image Deblurring ^{_{M. Ren, M. Delbracio, H. Talebi, G. Gerig, and P. Milanfar.}}	ICCV 2023	Paper/
2021/12	DVSR	Deblurring via Stochastic Refinement ^{_{Jay Whang, Mauricio Delbracio, Hossein Talebi, Chitwan Saharia, Alexandros G. Dimakis, Peyman Milanfar}}	CVPR 2022	Paper/Code

1.2.4 Dehaze

Release	Method	Title	Pub.	Link
2023/08	HazeAug	Frequency Compensated Diffusion Model for Real-scene Dehazing ^{_{Jing Wang, Songtao Wu, Kuanhong Xu, Zhiqiang Yuan}}	Neural Networks 2024	Paper/Code
2022/11	WeatherDiff	Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models ^{_{Ozan Özdenizci, Robert Legenstein}}	TPAMI 2023	Paper/Code

1.2.5 Low-light Enhancement

Release	Method	Title	Pub.	Link
2024/01	CFWD	Low-light Image Enhancement via CLIP-Fourier Guided Wavelet Diffusion ^{_{Minglong Xue, Jinhong He, Yanyi He, Zhipu Liu, Wenhai Wang, Mingliang Zhou}}	arXiv 2024	Paper/Code
2023/12	L²DM	L²DM: A Diffusion Model for Low-Light Image Enhancement ^{_{Lv, Xingguo and Dong, Xingbo and Jin, Zhe and Zhang, Hui and Song, Siyi and Li, Xuejun}}	PRCV 2023	Paper/Code
2023/11	Reti-Diff	Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model ^{_{Chunming He, Chengyu Fang, Yulun Zhang, Kai Li, Longxiang Tang, Chenyu You, Fengyang Xiao, Zhenhua Guo, Xiu Li}}	arXiv 2023	Paper/Code
2023/10	GASD	Global Structure-Aware Diffusion Process for Low-Light Image Enhancement ^{_{Jinhui Hou, Zhiyu Zhu, Junhui Hou, Hui Liu, Huanqiang Zeng, Hui Yuan}}	NIPS 2023	Paper/Code
2023/10	LLDE	LLDE: Enhancing Low-Light Images with Diffusion Model ^{_{Xin Peng Oo, Chee Seng Chan}}	ICIP 2023	Paper/Code
2023/09	-	Bootstrap Diffusion Model Curve Estimation for High Resolution Low-Light Image Enhancement ^{_{Jiancheng Huang, Yifan Liu, Shifeng Chen}}	PRICAI 2023	Paper/
2023/08	ExposureDiffusion	ExposureDiffusion: Learning to Expose for Low-light Image Enhancement ^{_{Yufei Wang, Yi Yu, Wenhan Yang, Lanqing Guo, Lap-Pui Chau, Alex C. Kot, Bihan Wen}}	ICCV 2023	Paper/Code
2023/08	Diff-Retinex	Diff-Retinex: Rethinking Low-light Image Enhancement with A Generative Diffusion Model ^{_{Xunpeng Yi, Han Xu, Hao Zhang, Linfeng Tang, Jiayi Ma}}	ICCV 2023	Paper/
2023/08	CLE Diffusion	CLE Diffusion: Controllable Light Enhancement Diffusion Model ^{_{Yuyang Yin, Dejia Xu, Chuangchuang Tan, Ping Liu, Yao Zhao, Yunchao Wei}}	MM 23	Paper/Code
2023/07	LLDiffusion	LLDiffusion: Learning Degradation Representations in Diffusion Models for Low-Light Image Enhancement ^{_{Tao Wang, Kaihao Zhang, Ziqian Shao, Wenhan Luo, Bjorn Stenger, Tae-Kyun Kim, Wei Liu, Hongdong Li}}	arXiv 2023	Paper/Code
2023/07	DiffLIE	DiffLIE: Low-Light Image Enhancment based on Deep Diffusion Model ^{_{Guanyu Wu; Cheng. Jin}}	ISCTIS 2023	Paper/
2023/06	-	Diffusion Model Based Low-Light Image Enhancement for Space Satellite ^{_{Yiman Zhu, Lu Wang, Jingyi Yuan, Yu Guo}}	arXiv 2023	Paper/
2023/05	PyDiff	Pyramid Diffusion Models For Low-light Image Enhancement ^{_{Dewei Zhou, Zongxin Yang, Yi Yang}}	IJCAI 2023	Paper/Code
2023/03	LPDM	Denoising Diffusion Post-Processing for Low-Light Image Enhancement ^{_{Savvas Panagiotou, Anna S. Bosman}}	arXiv 2023	Paper/Code
2023/03	DiD	Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognition ^{_{Cindy M. Nguyen, Eric R. Chan, Alexander W. Bergman, Gordon Wetzstein}}	WACV 2024	Paper/Code
2023/01	DiffLL	Low-Light Image Enhancement with Wavelet-based Diffusion Models ^{_{Hai Jiang, Ao Luo, Songchen Han, Haoqiang Fan, Shuaicheng Liu}}	TOG 2023	Paper/Code

1.2.6 Image Fusion

Release	Method	Title	Pub.	Link
2023/06	FusionDiff	FusionDiff: Multi-focus image fusion using denoising diffusion probabilistic models ^{_{Mining Li, Ronghao Pei, Tianyou Zheng, Yang Zhang, Weiwei Fu}}	ESWA 2024	Paper/Code
2023/03	DDFM	DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion ^{_{Zixiang Zhao, Haowen Bai, Yuanzhi Zhu, Jiangshe Zhang, Shuang Xu, Yulun Zhang, Kai Zhang, Deyu Meng, Radu Timofte, Luc Van Gool}}	ICCV 2023	Paper/Code
2023/01	Dif-Fusion	Dif-Fusion: Towards High Color Fidelity in Infrared and Visible Image Fusion with Diffusion Models ^{_{Jun Yue, Leyuan Fang, Shaobo Xia, Yue Deng, Jiayi Ma}}	TIP 2023	Paper/Code

2. Extended Diffusion Models In Low-level Vision

2.1 Diffusion Models In Low-level Medical Image Analysis

2.1.1 MRI

Release	Method	Title	Pub.	Link
2022/07	AdaDiff	Adaptive Diffusion Priors for Accelerated MRI Reconstruction ^{_{Alper Güngör, Salman UH Dar, Şaban Öztürk, Yilmaz Korkmaz, Gokberk Elmas, Muzaffer Özbey, Tolga Çukur}}	MedIA 2023	Paper/Code
2021/10	Score-MRI	Score-based diffusion models for accelerated MRI ^{_{Hyungjin Chung, Jong Chul Ye}}	MedIA 2022	Paper/Code
2021/08	CSGM	Robust Compressed Sensing MRI with Deep Generative Priors ^{_{Alper Güngör, Salman UH Dar, Şaban Öztürk, Yilmaz Korkmaz, Gokberk Elmas, Muzaffer Özbey, Tolga Çukur}}	NIPS 2021	Paper/Code

2.1.2 X-ray-based

Release	Method	Title	Pub.	Link
2022/11	DOLCE	DOLCE: A Model-Based Probabilistic Diffusion Framework for Limited-Angle CT Reconstruction ^{_{Jiaming Liu, Rushil Anirudh, Jayaraman J. Thiagarajan, Stewart He, K. Aditya Mohan, Ulugbek S. Kamilov, Hyojin Kim}}	ICCV 2023	Paper/Code

2.1.3 Multi-modal

Release	Method	Title	Pub.	Link
2023/04	FGDM	Zero-shot Medical Image Translation via Frequency-Guided Diffusion Models ^{_{Yunxiang Li, Hua-Chieh Shao, Xiao Liang, Liyuan Chen, Ruiqi Li, Steve Jiang, Jing Wang, You Zhang}}	TMI 2023	Paper/
2022/09	-	Conversion Between CT and MRI Images Using Diffusion and Score-Matching Models ^{_{Qing Lyu, Ge Wang}}	arXiv 2022	Paper/
2022/07	UMM-CSGM	A Novel Unified Conditional Score-based Generative Framework for Multi-modal Medical Image Completion ^{_{Xiangxi Meng, Yuning Gu, Yongsheng Pan, Nizhuan Wang, Peng Xue, Mengkang Lu, Xuming He, Yiqiang Zhan, Dinggang Shen}}	arXiv 2022	Paper/

2.1.4 Other Modalities

Release	Method	Title	Pub.	Link
2024/02	-	Dehazing Ultrasound using Diffusion Models ^{_{Tristan S.W. Stevens, Faik C. Meral, Jason Yu, Iason Z. Apostolakis, Jean-Luc Robert, Ruud J.G. Van Sloun}}	TMI 2024	Paper/
2022/09	PET-DDM	PET image denoising based on denoising diffusion probabilistic models ^{_{Kuang Gong, Keith A. Johnson, Georges El Fakhri, Quanzheng Li, Tinsu Pan}}	Eur. J. Nucl. Med. Mol. Imaging 2023	Paper/
2022/01	DenoOCT-DDPM	Unsupervised Denoising of Retinal OCT with Diffusion Probabilistic Model ^{_{Dewei Hu, Yuankai K. Tao, Ipek Oguz}}	SPIE 2022	Paper/Code

2.2 Diffusion Models In Remote Sensing For Low-level Vision Tasks

2.2.1 Visible-light Remote Sensing Image

Release	Method	Title	Pub.	Link
2023/10	EDiffSR	EDiffSR: An Efficient Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution ^{_{Yi Xiao, Qiangqiang Yuan, Kui Jiang, Jiang He, Xianyu Jin, Liangpei Zhang}}	TGRS 2024	Paper/Code
2023/09	RSDiff	RSDiff: Remote Sensing Image Generation from Text Using Diffusion Model ^{_{Ahmad Sebaq, Mohamed ElHelw}}	arXiv 2023	Paper/
2023/08	ARDD-Net	Remote Sensing Image Dehazing Using Adaptive Region-Based Diffusion Models ^{_{Y Huang, S Xiong}}	LGRS 2023	Paper/
2022/09	PSSR	Diffusion Model with Detail Complement for Super-Resolution of Remote Sensing ^{_{Liu, Jinzhe and Yuan, Zhiqiang and Pan, Zhaoying and Fu, Yiqun and Liu, Li and Lu, Bin}}	Remote Sensing 2022	Paper/

2.2.2 HSI

Release	Method	Title	Pub.	Link
2023/07	DDPM-Fus	Hyperspectral and Multispectral Image Fusion Using the Conditional Denoising Diffusion Probabilistic Model ^{_{Shuaikai Shi, Lijun Zhang, Jie Chen}}	arXiv 2023	Paper/Code
2023/07	-	A Noise-Model-Free Hyperspectral Image Denoising Method Based on Diffusion Model ^{_{Deng, Keli and Jiang, Zhongshun and Qian, Qipeng and Qiu, Yi and Qian, Yuntao}}	IGASS 2023	Paper/
2023/07	R2H-CCD	R2H-CCD: Hyperspectral Imagery Generation from RGB Images Based on Conditional Cascade Diffusion Probabilistic Models ^{_{Zhang, Lei and Luo, Xiaoyan and Li, Sen and Shi, Xiaofeng}}	IGASS 2023	Paper/
2023/03	DDS2M	DDS2M: Self-Supervised Denoising Diffusion Spatio-Spectral Model for Hyperspectral Image Restoration ^{_{Yuchun Miao, Lefei Zhang, Liangpei Zhang, Dacheng Tao}}	ICCV 2023	Paper/Code
2023/01	HSR-Diff	HSR-Diff:Hyperspectral Image Super-Resolution via Conditional Diffusion Models ^{_{Chanyue Wu, Dong Wang, Hanyu Mao, Ying Li}}	ICCV 2023	Paper/

2.2.3 SAR

Release	Method	Title	Pub.	Link
2023/07	-	Unsupervised SAR Despeckling Based on Diffusion Model ^{_{Xiao, Siyao and Huang, Libing and Zhang, Shunsheng}}	IGASS 2023	Paper/
2023/08	-	Diffusion Models for Interferometric Satellite Aperture Radar ^{_{Alexandre Tuel, Thomas Kerdreux, Claudia Hulbert, Bertrand Rouet-Leduc}}	arXiv 2023	Paper/Code
2023/06	-	SAR Despeckling using a Denoising Diffusion Probabilistic Model ^{_{Malsha V. Perera, Nithin Gopalakrishnan Nair, Wele Gedara Chaminda Bandara, Vishal M. Patel}}	LGRS 2023	Paper/Code

2.2.4 Multi-modal

Release	Method	Title	Pub.	Link
2023/07	-	Improved Flood Insights: Diffusion-Based SAR to EO Image Translation ^{_{Minseok Seo, Youngtack Oh, Doyi Kim, Dongmin Kang, Yeji Choi}}	arXiv 2023	Paper/
2023/04	-	Cloud Removal in Remote Sensing Using Sequential-Based Diffusion Models ^{_{Zhao, Xiaohu and Jia, Kebin}}	Remote Sensing 2023	Paper/
2023/04	DDRF	DDRF: Denoising Diffusion Model for Remote Sensing Image Fusion ^{_{ZiHan Cao, ShiQi Cao, Xiao Wu, JunMing Hou, Ran Ran, Liang-Jian Deng}}	arXiv 2023	Paper/
2023/03	DDPM-CR	Denoising Diffusion Probabilistic Feature-Based Network for Cloud Removal in Sentinel-2 Imagery ^{_{Jing, Ran and Duan, Fuzhou and Lu, Fengxian and Zhang, Miao and Zhao, Wenji}}	Remote Sensing 2023	Paper/

2.3 Varied Low-level Vision Tasks In Video Through Diffusion Models

2.3.1 Video Frame Prediction and Interpolation

Release	Method	Title	Pub.	Link
2023/03	LDMVFI	LDMVFI: Video Frame Interpolation with Latent Diffusion Models ^{_{Duolikun Danier, Fan Zhang, David Bull}}	arXiv 2023	Paper/Code
2022/06	RaMViD	Diffusion Models for Video Prediction and Infilling ^{_{Tobias Höppe, Arash Mehrjou, Stefan Bauer, Didrik Nielsen, Andrea Dittadi}}	TMLR 2022	Paper/
2022/05	MCVD	MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation ^{_{Vikram Voleti, Alexia Jolicoeur-Martineau, Christopher Pal}}	NIPS 2022	Paper/Code
2022/03	RVD	Diffusion Probabilistic Modeling for Video Generation ^{_{Ruihan Yang, Prakhar Srivastava, Stephan Mandt}}	Entropy 2023	Paper/Code

2.3.2 Super Resolution For Video Generation

Release	Method	Title	Pub.	Link
2024/03	SATeCo	Learning Spatial Adaptation and Temporal Coherence in Diffusion Models for Video Super-Resolution ^{_{Zhikai Chen, Fuchen Long, Zhaofan Qiu, Ting Yao, Wengang Zhou, Jiebo Luo, Tao Mei}}	CVPR 2024	Paper/
2024/01	-	Inflation with Diffusion: Efficient Temporal Adaptation for Text-to-Video Super-Resolution ^{_{Xin Yuan, Jinoo Baek, Keyang Xu, Omer Tov, Hongliang Fei}}	WACV 2024	Paper/
2023/09	LAVIE	LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models ^{_{Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, Yuwei Guo, Tianxing Wu, Chenyang Si, Yuming Jiang, Cunjian Chen, Chen Change Loy, Bo Dai, Dahua Lin, Yu Qiao, Ziwei Liu}}	arXiv 2023	Paper/Code
2023/05	PYoCo	Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models ^{_{Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming-Yu Liu, Yogesh Balaji}}	ICCV 2023	Paper/Demo
2023/04	-	Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models ^{_{Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, Karsten Kreis}}	CVPR 2023	Paper/Demo

2.3.3 Video Restoration

Release	Method	Title	Pub.	Link
2024/03	DiffTTA	Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal ^{_{Yijun Yang, Hongtao Wu, Angelica I. Aviles-Rivero, Yulun Zhang, Jing Qin, Lei Zhu}}	CVPR 2024	Paper/Code

Related Surveys Recommended

Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectives
arXiv 2024. [Paper]
Apr. 2024

Diffusion Models, Image Super-Resolution And Everything: A Survey
arXiv 2024. [Paper]
Jan. 2024

State of the Art on Diffusion Models for Visual Computing
arXiv 2023. [Paper]
Oct. 2023

Diffusion Models for Image Restoration and Enhancement -- A Comprehensive Survey.
arXiv 2023. [Paper]
Aug. 2023

Survey on Diverse Image Inpainting using Diffusion Models
PCEMS 2023. [Paper]
Jun. 2023

Diffusion Models for Medical Image Analysis: A Comprehensive Survey
Medical Image Analysis 2023. [Paper]
Nov. 2022

Diffusion Models in Vision: A Survey
TPAMI 2023. [Paper]
Sep. 2022

Diffusion Models: A Comprehensive Survey of Methods and Applications
ACM Computing Surveys 2023. [Paper]
Sep. 2022

Datasets for low-level vision tasks

Due to space limitations, we provide a summary of commonly used datasets for several classical natural low-level vision tasks here, including their scales, sources, modalities, and remarks. Clicking on the dataset will redirect you to its download link.

Tasks	Datasets	Scales	Sources	Modalities	Remarks
SR	BSD500	500	TPAMI 2010	Syn	A synthetic benchmark that is initially designed for object contour detection.
SR	Set14	14	TPAMI 2015	Syn	Commonly utilized for testing performance of super-resolution algorithms.
SR	Manga109	109	MTAP 2015	Syn	Compiled mainly for academic research on Japanese manga media processing.
SR	General100	100	ECCV 2016	Syn	Synthesized images in uncompressed BMP format covering various scales.
SR	DIV2K	900/100	NTIRE 2018	Real	A commonly-used dataset with diverse scenarios and realistic degradations.
SR	Flickr1024	1024	ICCVW 2019	Syn	A large-scale stereo image dataset with high-quality pairs and diverse scenarios.
SR	Urban100	100	CVPR 2019	Syn	Sourced from urban environments: city streets, buildings, and urban landscapes.
SR	DRealSR	31970	ECCV 2020	Real	Benchmarks captured by DSLR cameras, circumventing simulated degradation.
Deblur	GoPro	2103/1111	CVPR 2017	Syn	Acquired by high-speed cameras for video quality assessment and restoration.
Deblur	HIDE	8422	ICCV 2019	Syn	Cover long-distance and short-distance scenarios degraded by motion blur.
Deblur	REDS	270/30	NTIRE 2019	Real	Contain 300 video sequences with dynamic duration and varied resolutions.
Deblur	BSD	80/20	ECCV 2020	Real	Comprise more scenes and use the proposed beam-splitter acquisition system.
Deblur	RealBlur	3758/980	ECCV 2020	Real	Cover common instances of motion blur, captured in raw and JPEG formats.
Dehaze	I-Haze	35	NTIRE 2018	Real	Indoor dataset with real haze for objective image dehazing and evaluation.
Dehaze	O-Haze	45	NTIRE 2018	Real	Outdoor dataset with real haze for objective image dehazing and evaluation.
Dehaze	Dense-Haze	33	ICIP 2019	Real	Real-world dataset with dense haze for robust single image dehazing methods.
Dehaze	RESIDE	13000/990	TIP 2019	Syn+Real	Divided into five subsets to highlight diverse sources and heterogeneous contents.
Dehaze	NH-Haze	55	CVRPW 2020	Real	The first non-homogeneous dehazing dataset with realistic haze distribution.
Dehaze	Haze-4K	4000	MM 2021	Syn	A large-scale synthetic dataset for image dehazing with varing distributions.
LLIE	MIT-Fivek	4500/500	CVPR 2011	Syn	A curated dataset of RAW photos adjusted by skilled retouchers for visual appeal.
LLIE	LOLv1	485/15	BMVC 2018	Real	The first dataset with image pairs from real scenarios for low-light enhancement.
LLIE	SID	5094	CVPR 2018	Real	A dataset of raw short-exposure images with their long-exposure reference images.
LLIE	SICE	589	TIP 2018	Syn	A large-scale multi-exposure image dataset with complex illumination conditions.
LLIE	ExDark	7363	CVIU 2019	Real	Collected in low-light scenarios with 12 classes and instance-level annotations.
LLIE	LOLv2-Real	689/100	TIP 2021	Real	A three-step shooting strategy is used to eliminate intra-pair image misalignments.
LLIE	LOLv2-Syn	900/100	TIP 2021	Syn	Synthetic dark images mimic real low-light photography via histogram analysis.
LLIE	SDSD-Indoor	62/6	ICCV 2021	Real	Indoor dataset collected from dynamic scenes under varying lighting conditions.
LLIE	SDSD-Outdoor	116/10	ICCV 2021	Real	Outdoor dataset collected from dynamic scenes under varying lighting conditions.
Derain	Rain100H	1800/100	CVPR 2017	Syn	Comprise synthetic datasets with five types of rain streaks for rain removal.
Derain	RainDrop	861/239	CVPR 2018	Syn	Image pairs with raindrop degradation, captured using the setup of dual glasses.
Derain	SPA-Data	638492/1000	CVPR 2019	Real	Design a semi-automatic method to generate clean images from real rain streaks.
Derain	MPID	3961/419	CVPR 2019	Syn+Real	A large-scale benchmark that focuses on driving and surveillance scenarios.
Derain	RainCityscapes	9432/1188	CVPR 2019	Syn	A famous rain removal dataset with paired depth maps for outdoor scenarios.
Derain	RainDS	3450/900	CVPR 2021	Syn+Real	A hybrid dataset with both real and synthesized data under diverse scenarios.
Derain	RainDirection	2920/430	ICCV 2021	Syn	A large-scale synthetic rainy dataset with directional labels in the training phase.
Derain	GT-RAIN	28217/2100	ECCV 2022	Real	The first paired derain dataset with real data by controlling non-rain variations.
Desnow	Snow100k	100000	TIP 2018	Syn+Real	A large-scale dataset with over 1k real-world images degraded by heavy snow.
Desnow	SRRS	16000	ECCV 2020	Syn+Real	A hybrid snow dataset with 15k synthesized images and 1k real-world images.
Desnow	CSD	10000	ICCV 2021	Syn	A large-scale desnowing dataset to comprehensively simulate snow scenarios.