A dive into the magical world of deep learning, unlocking the artistic capabilities of your machine.
"We keep moving forward, opening new doors, and doing new things, because we're curious and curiosity keeps leading us down new paths." --- Walt Disney
Machine learning is an integral part of our life. It's everywhere from the thing that lives on the top of our phone's keyboard to the things that drive the field of self-driving cars. The area of machine learning is advancing day-after-day, with innovations and new ideas coming forward. One such contribution of Machine learning, precisely the field of deep learning is GAN --- Generative Adversarial Networks, something which in my opinion is a pure form of magic which is created by science.
GAN is a class of deep learning framework dedicated to creating new things. Unlike conventional deep learning techniques that are used to detect various things, GAN is used to produce new things. Think of GAN as a painter; it likes to paint. Identifying a particular item, say a hat is something any literate human can do. We can easily mimic this on a computer using a deep learning framework. But, GAN, on the other hand, is a step towards creativity, a level close to intelligence.
Credit: ThisPersonDoesNotExist
The image shown above is not of a real person but is generated by a machine from scratch. That is what GAN is all about.
GAN was proposed by Ian Goodfellow and his colleagues in the year 2014 when Ian was about 28. Being the father of GAN, he is considered as the man who gave machines the gift of imagination. The legends go that Ian was approached by his friends regarding a *machine that produces images *at a bar one night. Now, let's have a closer look at Ian's GANs.
Before we get a rough idea about how GANs work, I request you to read a little bit about deep learning, all it's general algorithms such as backpropagation and CNN to understand GANs better. Here are some useful links.
1. Backpropagation [http://neuralnetworksanddeeplearning.com/chap2.html]
2. CNN [https://brohrer.github.io/how_convolutional_neural_networks_work.html]
3. Adaptive Deconvolutional Networks [https://ieeexplore.ieee.org/document/6126474]
The two main pillars of GAN are generator and discriminator. These are two sophisticated neural networks that are the working hands behind the magic of GAN.
Let's first try to understand these with an example.
Consider the generator and discriminator to be a student and teaching assistant (TA) respectively. Both of them are present here for an examination. Let's assume both of them know nothing in the beginning. The job of the student (generator) is to write the answers, and the role of the TA (discriminator) is to check the answers from an answer book. Initially, the students scribble out a few random answers and pass that on to the TA. The TA randomly allocates rough marks to those answers, and then he goes on to check the answers with the answer book. In doing so, The TA realizes his mistakes and learns from them while giving the correct marks.
Furthermore, once the students get the feedback of the marks, he learns about his mistakes and tries to write better, close to the correct answer. This cycle of double looping continues until the answers produced are really close to the right answers. This, in simple terms, is how GAN works.
Generative Adversarial Network framework by Thalles Silva
As mentioned above, the Generator and Discriminator are both neural networks. The generator, as the name suggests, generates new data instances; on the other hand, discriminator acts as a judge for the generated data instances. Both of these networks are trained from scratch, while they compete with each other in a game of guesses. The game is based on the criteria of data distributions. We won't dive deep into the mathematics of their working. The generator generates fake images from a sample random noise (to be honest, nothing is truly random, so let's call it pseudo-random) and then it passes that on to the discriminator in the hopes that those fake images get accepted as real images by the discriminator. The discriminator is also passed with ground-truth, i.e. real classified dataset. The discriminator tries to identify the real and the fake photos, by firstly giving random probabilities between 0 (fake) and 1 (real) to the images. Then, it learns from its mistakes and back-propagates the error to provide better probabilities. This cycle continues until it gives close to correct probabilities to all the images, i.e. close to 1 for real pictures and close to 0 for fake images. Once this is done, the feedback of the probabilities of the fake images is backpropagated to the generator, which then tries to create new images with better probabilities. This double-cycle continues until the likelihood of the generated images gets close to 1. This, in a nutshell, is how GAN works.
Credit: Generative Adversarial Nets
- The generator generates images from random noise and sent it forward.
- Discriminator gives probabilities of the images received.
- Discriminator learns from the ground-truth dataset and provides more accurate probabilities. Then, it returns it backwards.
- Generator learns from the returned probabilities and tries to create images with better probabilities.
Credits: CycleGAN
Now, let's talk about some fun and interesting real-life applications of GAN; this will help you realize how magical GANs really are.
Here is a list of some fantastic applications of GAN, listed by Jason Brownlee. Please go through his article to learn about all these applications.
- Generate Examples for Image Datasets
- Generate Photographs of Human Faces
- Generate Realistic Photographs
- Generate Cartoon Characters
- Image-to-Image Translation
- Text-to-Image Translation
- Semantic-Image-to-Photo Translation
- Face Frontal View Generation
- Generate New Human Poses
- Photos to Emojis
- Photograph Editing
- Face Aging
- Photo Blending
- Super Resolution
- Photo Inpainting
- Clothing Translation
- Video Prediction
- 3D Object Generation
You can see GAN in action on this amazing website, thispersondoesnotexist.com. This website uses StyleGAN2 and presents to you a computer-generated random picture of a person who doesn't exist in real life whenever you visit this website. Just keep on refreshing. The images generated are shockingly realistic, and this helps to show the real power of GANs. This website was created by Phillip Wang to showcase the potential of StyleGAN2 and GANs in general. Isn't this amazing?!
Now, that we know what GAN is capable of, let's try to code our own DCGAN in python.
A major thanks to Hadelin de Ponteves.
Firstly, you'll need the dataset consisting of colour images. Please visit this link to download the *CIFAR-10 python version *dataset.
https://www.cs.toronto.edu/~kriz/cifar.html
Extract the dataset and put it in folder 'data'. Also, create an empty folder in the same directory to hold the output images. Start a new python file in the directory containing the data and the empty folder.
Note: If you don't want to manually download the dataset or you're having some problem in downloading it, then please don't worry. The fourth code cell in the following Colab file will do it for you.
Tip: You can code the complete DCGAN on Google Colab. In your Goggle Colab notebook, go to Runtime > Change Runtime Type and under hardware accelerator select GPU.
Google Colab Notebook link --- https://colab.research.google.com/drive/1iHnpkzUcnm_vm-A-Z_LWXS4c9lNva9Ez?usp=sharing
1. Generative Adversarial Networks [https://arxiv.org/abs/1406.2661]
2. Backpropagation [http://neuralnetworksanddeeplearning.com/chap2.html]
3. CNN [https://brohrer.github.io/how_convolutional_neural_networks_work.html]
4. Adaptive Deconvolutional Networks [https://ieeexplore.ieee.org/document/6126474]
5. Computer Vision & GANs [https://www.superdatascience.com/courses/computer-vision-z-learn-opencv-gans-cutting-edge-ai]
1. Andrew Y. Ng & Michael I. Jordan
2. Deep Convolutional Generative Adversarial Networks
3. StyleGAN
4. CycleGAN