This is the official implementation of paper title "Beyond Joint Demosaicking and Denoising: An Image Processing Pipeline for a Pixel-bin Image Sensor" from CVPRW21. To download full paper [Click Here].
Please consider to cite this paper as follows:
@inproceedings{a2021beyond,
title={Beyond joint demosaicking and denoising: An image processing pipeline for a pixel-bin image sensor},
author={Sharif, SMA and Naqvi, Rizwan Ali and Biswas, Mithun},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={233--242},
year={2021}
}
Paper title: SAGAN: Adversarial Spatial-asymmetric Attention for Noisy Nona-Bayer Reconstruction
[Paper Link] | [Supplemental] | [Code]
Please do consider citing SAGAN as follows:
@inproceedings{a2021beyond,
title={SAGAN: Adversarial Spatial-asymmetric Attention for Noisy Nona-Bayer Reconstruction},
author={Sharif, SMA and Naqvi, Rizwan Ali and Biswas, Mithun},
booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
pages={0-0},
year={2021}
}
BJDD tackle the challenges of joint demosaicing and denoising (JDD) of artefact prone non-Bayer Quad-Bayer CFA and Bayer CFA pattern. Notably, such CFA pair is substantially leveraging in the recent smartphone cameras (denoted as Pixel-bin image sensor).
Figure: Commonly used CFA patterns of pixel-bin image 086 sensors. Left : Quad-Bayer CFA. Right: Bayer-CFA.
Figure: Overview of the proposed method, including network architecture and submodules.
Figure: Quantitative comparison between proposed method and existing JDD methods for Quad-Bayer reconstruction.
Python 3.8
CUDA 10.1 + CuDNN
pip
Virtual environment (optional)
Please consider using a virtual environment to continue the installation process.
git clone https://github.com/sharif-apu/BJDD_CVPR21.git
cd BJDD_CVPR21
pip install -r requirement.txt
** [Click Here]** to download pretrained weights and save it to weights/ directory for inferencing with Quad-bayer CFA
python main.py -i
A few testing images are provided in a sub-directory under testingImages (i.e., testingImages/sampleImages/)
In such occasion, reconstructed image(s) will be available in modelOutput/sampleImages/.
To inference with custom setting execute the following command:
python main.py -i -s path/to/inputImages -d path/to/outputImages -ns=sigma(s)
Here,-ns specifies the standard deviation of a Gaussian distribution (i.e., -ns=5, 10, 15),-s specifies the root directory of the source images
(i.e., testingImages/), and -d specifies the destination root (i.e., modelOutput/).
To start training we need to sampling the images according to the CFA pattern and have to pair with coresponding ground-truth images. To sample images for pair training please execute the following command:
python main.py -ds -s /path/to/GTimages/ -d /path/to/saveSamples/ -g 2 -n 10000
Here -s flag defines your root directory of GT images, -d flag defines the directory where sampled images should be saved, and -g flag defines the binnig factr (i.e., 1 for bayer CFA, 2 for Quad-bayer), -n defines the number of images have to sample (optional)
After extracting samples, please execute the following commands to start training:
python main.py -ts -e X -b Y
To specify your trining images path, go to mainModule/config.json and update "gtPath" and "targetPath" entity.
You can specify the number of epoch with -e flag (i.e., -e 5) and number of images per batch with -b flag (i.e., -b 12).
For transfer learning execute:
python main.py -tr -e -b
We also trained our model with Bayer CFA. To download pretrained Bayer weights [Click Here]. In such occasion, please update binning factor entity in mainModule/config.json file.
Check model configuration:
python main.py -ms
Create new configuration file:
python main.py -c
Update configuration file:
python main.py -u
Overfitting testing
python main.py -to
For any further query, feel free to contact us through the following emails: sma.sharif.cse@ulab.edu.bd, rizwanali@sejong.ac.kr, or mithun.bishwash.cse@ulab.edu.bd