This is a Tensorflow implementation of our method to generate diverse images from semantic layouts.
Diverse Image Synthesis from Semantic Layouts via Conditional IMLE
Ke Li*, Tianhao Zhang*, Jitendra Malik
(* equal contribution, alphabetical order)
In ICCV 2019.
Required python version: Python 2.7
Required python libraries: Tensorflow (>=1.0) + Scipy + Numpy + Pillow.
Tested in Ubuntu + Intel i7 CPU + Nvidia Titan X (Pascal) with Cuda (>=8.0) and CuDNN (>=5.0). CPU mode should also work with minor changes.
- Clone this repository.
- Download the VGG19 pretrained model by running "python download_models.py".
- Download the pretrained model from here and extract at the root directory
- Run "test.py" to synthesize images.
- The synthesized images are saved in "gta_demo/result/"
You can download the pre-processed dataset from here or you can download it from the official website and run "python preprocess.py" to process the images.
We train our model based on the CRN pretrained model. The pre-processed model (extra channels are added) can be downloaded from here. You can also download the CRN pretrained model from their project and then preprocess the model by running "python preprocess_crn_model.py".
Once you have downloaded the dataset, you can generate the rarity mask (for loss rebalancing) and rarity bins (for dataset rebalancing) by running "python gen_dataset_weight.py" or you can download the pre-generated ones from here
Run "python train.py" to start training
If you have any question or request about the code and data, please email me at bryanzhang97@gmail.com.
If you find this useful for your research, please use the following.
@inproceedings{li2019diverse,
title={Diverse Image Synthesis from Semantic Layouts via Conditional IMLE},
author={Ke Li and Tianhao Zhang and Jitendra Malik},
booktitle={Proceedings of the IEEE international conference on computer vision},
year={2019}
}
This code borrows heavily from Photographic Image Synthesis with Cascaded Refinement Networks.
MIT License