Try-First

The following project is an implementation of the paper "VITON: An Image-based Virtual Try-on Network" from University of Maryland, College Park, China. https://arxiv.org/abs/1711.08447

Table of contents

• Introduction
• Implementation Details
• Training Process
• Installation

Introduction :-

TryFirst is an interactive web-app which benefits users by allowing them to try-on a particular cloth to see how it suits them. Thus, allowing consumers to virtually try on clothes will not only enhance their shopping experience, transforming the way people shop for clothes, but also save cost for retailer.

Implemenetation_details

• Pose Generation using openpose.
• Parser are generated using SS-nan (opensource pre-trained model) .
• Used Generative Adversarial Networks with Adversarial loss, perceptual loss and L1 loss for smoothening.
• Used U-Net for generator and a downsampler for discriminator.

Training Process

• The pose generated and parser and person image are concatenated along with and cloth imgae are fedded to GMM.
• output of above is a warped cloth.
• Now that concatenated image along with warped cloth is feeded to Gans.
• Final output is image of person wearing desired cloth.
• Final loss of generator on validation : 3.62001
• Final loss of discriminator on validation :0.003821
• Epochs Trained : 50
• Optimizer : Ranger

Installation

Prerequisites

PIL
PyTorch
TorchVision
tqdm

Person representation extraction

The person representation used in this project are extracted by a 2D pose estimator and a human parser:

• Realtime Multi-Person Pose Estimation
• Self-supervised Structure-sensitive Learning

Download the Model files from the links provided below in the appropriate locations.

1. TOM Model
2. GMM Model
3. Pose Vector Model
4. Pose Parser Model

Download repository

$ git clone https://github.com/JaZz-9/Try-First

After adding the models, run the following command:

1. Installing all the pre-requisite libraries.

pip install -r requirements.txt

2. Executing model using streamlit script

streamlit run app.py

Outputs

References

[1] HAN X., WU Z., WU Z., YU R., DAVIS L. S.: Viton: An image-based virtual try-on network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018).
[2] BROCK A., DONAHUE J., SIMONYAN K.: Large scale GAN training for high fidelity natural image synthesis. In International Conference on Learning Representations (2019).
[3] CAO Z., SIMON T., WEI S.-E., SHEIKH Y.: Realtime multiperson 2d pose estimation using part affinity fields. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017).
[4] GONG K., LIANG X., ZHANG D., SHEN X., LIN L.: Look into person: Self-supervised structure-sensitive learning and a new benchmark for human parsing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017).
[5] KARRAS T., LAINE S., AILA T.: A style-based generator architecture for generative adversarial networks. arXiv preprint arXiv:1812.04948 (2018).
[6] WANG B., ZHENG H., LIANG X., CHEN Y., LIN L., YANG M.: Toward characteristic-preserving image-based virtual try-on network. In Proceedings of the European Conference on Computer Vision (2018).