README_paper.md

The code for reproducing the ImageNet results in the ICLR2018 papers; spectral normalization and projection discrimiantor

Official Chainer implementation for reproducing the results of conditional image generation on ILSVRC2012 dataset (ImageNet) with spectral normalization and projection discrimiantor.

References

• Takeru Miyato, Toshiki Kataoka, Masanori Koyama, Yuichi Yoshida. Spectral Normalization for Generative Adversarial Networks. ICLR2018. OpenReview
• Takeru Miyato, Masanori Koyama. cGANs with Projection Discriminator. ICLR2018. OpenReview

Setup

Install OpenMPI and NCCL (required for multi-GPU training with ChainerMN)

Please see the following installation guide: https://chainermn.readthedocs.io/en/latest/installation/guide.html#requirements

(Note: we provide the single GPU training code here, but we have not checked the peformance of the models trained on single GPU. All of the results showed in the papers are produced by the models trained on 4 GPUs)

Install required python libraries:

pip install -r requirements_paper.txt

Additionaly we recommend to install the latest cupy:

pip uninstall cupy
git clone https://github.com/cupy/cupy.git
cd cupy
python setup.py install

Download ImageNet dataset:

Please download ILSVRC2012 dataset from http://image-net.org/download-images

Preprocess dataset:

cd datasets
IMAGENET_TRAIN_DIR=/path/to/imagenet/train/ # path to the parent directory of category directories named "n0*******".
PREPROCESSED_DATA_DIR=/path/to/save_dir/
bash preprocess.sh $IMAGENET_TRAIN_DIR $PREPROCESSED_DATA_DIR
# Make the list of image-label pairs for all images (1000 categories, 1281167 images).
python imagenet.py $PREPROCESSED_DATA_DIR
# (optional) Make the list of image-label pairs for dog and cat images (143 categories, 180373 images).
python imagenet_dog_and_cat.py $PREPROCESSED_DATA_DIR

Download inception model:

python source/inception/download.py --outfile=datasets/inception_model

Training

Spectral normalization + projection discriminator for 128x128 all ImageNet images:

LOGDIR=/path/to/logdir/
CONFIG=configs/sn_projection_dog_and_cat.yml
# multi-GPU
mpiexec -n 4 python train_mn.py --config=configs/sn_projection.yml --results_dir=$LOGDIR --data_dir=$PREPROCESSED_DATA_DIR
# single-GPU
python train.py --config=$CONFIG --results_dir=$LOGDIR --data_dir=$PREPROCESSED_DATA_DIR

• pretrained models
• generated images at 450K iterations (Inception score:29.7, Intra class FID:103.1)
• generated images at 850K iterations (Inception score:36.8, Intra class FID:92.4)
• Examples of generated images at 450K iterations:

Spectral normalization + concat discriminator for 128x128 all ImageNet images:

LOGDIR=/path/to/logdir/
CONFIG=configs/sn_projection_dog_and_cat.yml
# multi-GPU
mpiexec -n 4 python train_mn.py --config=configs/sn_concat --results_dir=$LOGDIR --data_dir=$PREPROCESSED_DATA_DIR
# single-GPU
python train.py --config=$CONFIG --results_dir=$LOGDIR --data_dir=$PREPROCESSED_DATA_DIR

• pretrained models
• generated images at 450K iterations (Inception score:21.1, Intra class FID:141.2)

(optional) Spectral normalization + projection discriminator for 128x128 dog and cat images:

LOGDIR=/path/to/logdir/
CONFIG=configs/sn_projection_dog_and_cat.yml
# multi-GPU
mpiexec -n 4 python train_mn.py --config=configs/sn_projection_dog_and_cat.yml --results_dir=$LOGDIR --data_dir=$PREPROCESSED_DATA_DIR
# single-GPU
python train.py --config=$CONFIG  --results_dir=$LOGDIR --data_dir=$PREPROCESSED_DATA_DIR

• pretrained models
• generated images (Inception score: 28.2)

Evaluation examples

(If you want to use pretrained models for the image generation, please download the model from link and set the snapshot argument to the path to the downloaded pretrained model file (.npz).)

Generate images

python evaluations/gen_images.py --config=$CONFIG --snapshot=${LOGDIR}/ResNetGenerator_<iterations>.npz --results_dir=${LOGDIR}/gen_images

Generate category morphing images

Regarding the index-category correspondence, please see 1K ImageNet or 143 dog and cat ImageNet.

python evaluations/gen_interpolated_images.py --n_zs=10 --n_intp=10 --classes $CATEGORY1 $CATEGORY2 --config=$CONFIG --snapshot=${LOGDIR}/ResNetGenerator_<iterations>.npz --results_dir=${LOGDIR}/gen_morphing_images

Calculate inception score (with the original OpenAI implementation)

python evaluations/calc_inception_score.py --config=$CONFIG --snapshot=${LOGDIR}/ResNetGenerator_<iterations>.npz --results_dir=${LOGDIR}/inception_score --splits=10 --tf