Image Compression using VAEGANs
Explore the repository»
View Report
tags : image compression, vaegans, gans, generative networks, flickr30k, deep learning, pytorch
In the modern world, a huge amount of data is being sent across the internet every day. Efficient data compression and communication protocols are of great research interest today. We focus on the compression of images and video (sequence of image frames) using deep generative models and show that they achieve a better performance in compression ratio and perceptual quality. We explore the use of VAEGANs for this task. We know that generative models such as GANs and VAE can reproduce an image from its latent vector. We ask the question whether we can go the other direction — from an image to a latent vector. Research, though limited, has shown that these types of methods are quite effective and efficient. A detailed description of algorithms and analysis of the results are available in the Report.
This project was built with
- python v3.8.5
- PyTorch v1.7
- The environment used for developing this project is available at environment.yml.
Clone the repository into a local machine and enter the src directory using
git clone https://github.com/vineeths96/Variational-Generative-Image-Compression
cd Variational-Generative-Image-Compression/srcCreate a new conda environment and install all the libraries by running the following command
conda env create -f environment.ymlThe dataset used in this project (Flickr30K) will be automatically downloaded and setup in input directory during execution.
To train the model with n channels run,
python main.py --train-model True --num-channels <n> This trains the VAEGAN model and saves it in the models directory.
To evaluate the model on the compressed images run,
python main.py This generates a folder in the results directory for each run. The generated folder contains the compressed images using different number of channels. This also calculates the average PSNR and SSIM values across different runs, and generates avg_psnr.txt and avg_ssim.txt in the results directory.
The architecture of the model is shown below. We freeze the GAN model and optimize for the best latent vector using gradient descent.
We evaluate the models on the Structural Similarity Index (SSIM) and Peak Signal to Noise Ratio (PSNR) between the original image and reconstructed image. More detailed results and inferences are available in report here.
| Number of Channel | SSIM | PSNR | CR |
|---|---|---|---|
| 28 Channels | 0.83 | 24.79 | 1.74 × |
| 16 Channels | 0.80 | 24.37 | 3.06 × |
| 8 Channels | 0.81 | 24.06 | 6.12 × |
| 4 Channels | 0.79 | 23.57 | 12.24 × |
| 2 Channels | 0.71 | 21.57 | 24.49 × |
Figure below compares the image quality of reconstruction for a sample image for different schemes.
Distributed under the MIT License. See LICENSE for more information.
Vineeth S - vs96codes@gmail.com
Project Link: https://github.com/vineeths96/Variational-Generative-Image-Compression