Neural Style Transfer (with app)

This repository includes a Pytorch implementation of the paper "Image Style Transfer Using Convolutional Neural Networks", Leon A. Gatys et al. that can run locally with an app (app.py -developed with Streamlit) or by modifying the input arguments of a script (main.py).

Brief description

Neural Style Transfer (NST) is a deep learning technique that generates an image based on the content of one image (the content image) and the style of another image (the style image).

Have a look at the next example generated with this repository. The content and style images are located at the top-left and top-right corners respectively, and the bottom row contains the output (stylized) image.

Installation with conda

These instructions assume that Anaconda or Miniconda are installed in the computer.

1. Open Anaconda Prompt and clone this repository in the desired path, and enter to it:

   cd <your_custom_directory_name>
   git clone https://github.com/javiertg9/neural-style-transfer-app
   cd neural-style-transfer-app

2. Create the environment with the needed dependencies with the provided environment.yaml file of this repo:

   conda env create -f environment.yml

3. Activate the environment:

   conda activate nst-app

How to use it

There are two ways to run this repository. The first one is through the app (app.py) done with Streamlit:

1) Running the app

You can run it simply by typing in the Anaconda Prompt the following command:

Streamlit run app.py

Then, your default browser should open locally the following page:

Here it is a video showing how you can use the demo. Its configuration relies on the parameters for performing NST that are located at the left sidebar:

• Weights of the loss function,
• number of iterations of the optimization process,
• and checkbox to save or not the resulting stylized image.

video_browser_edited.mp4

2) Running the script

You can run the default images/ parameter values by using the next command:

python main.py

Doing so, there will appear the following triplet of openCV windows:

To stop the optimization process before it finishes just press in the keyboard q. This will stop the execution and save, if specified, the resulting image.

But you can also change the images or parameter values and decide if you want to save the resulting image in the data/ouput-images folder.
These are the parameter specifications that appear after executing python main.py --help:

[Clik to expand and see the optional parameters]

usage: main.py [-h] [--content_img_name CONTENT_IMG_NAME]
               [--style_img_name STYLE_IMG_NAME]
               [--content_weight CONTENT_WEIGHT] [--style_weight STYLE_WEIGHT]
               [--tv_weight TV_WEIGHT] [--model {vgg16,vgg19}]
               [--save_stylized_image] [--niter NITER]

optional arguments:
  -h, --help            show this help message and exit
  --content_img_name CONTENT_IMG_NAME
                        name and extension of the CONTENT image located at the
                        "data/content-images" folder. For example: lion.jpg
  --style_img_name STYLE_IMG_NAME
                        name and extension of the STYLE image located at the
                        "data/style-images" folder. For example: wave.jpg
  --content_weight CONTENT_WEIGHT
                        weight (importance) of the CONTENT image in the
                        resulting stylized image
  --style_weight STYLE_WEIGHT
                        weight (importance) of the STYLE image in the
                        resulting stylized image
  --tv_weight TV_WEIGHT
                        The higher value of this weight, the higher degree of
                        smoothness in the stylized image
  --model {vgg16,vgg19}
                        Select which VGG model (vgg16 or vgg19) to use to
                        define the perceptual losses (recommendation: choose
                        vgg19 as it offers better results)
  --save_stylized_image
                        write this flag if you want to save the resulting
                        stylized image
  --niter NITER         Number of iterations to perform during the
                        optimization process

For example, if we execute the following command:

python main.py --content_img_name=bridge.jpeg --style_img_name=vangogh2.jpg --content_weight=0.001 --niter=40 --save_stylized_image

then the result will be:

and we would get in data/ouput-images the image called cbridge_svangogh2.jpg:

Useful resources

To do this implementation I found the following works to be highly useful:

• NST YouTube playlist of AI epiphany, and its related repository: pytorch-neural-style-transfer,
• the paper "Very Deep Convolutional Netowrks for Large-Scale Image Recognition", Simonyan et al., for understanding the VGG-net architecture,
• Pytorch and Torchvision implementations of the VGG-net,
• and of course the original paper.

The images used in this repo have been mainly obtained from these sources:

• V. Van Gogh paintings from rawpixel,
• images from this repo,
• bridge pic.