Hyperparameters

[RiaanZoetmulder]

Hello,

I have some questions regarding hyperparameters. I have made my own implementation in tensorflow. My question is what are your exact hyperparameter specifications for getting good results?

Currently I am using:
lr (for all) --> 0.0003
l2_reg --> 0.0
batch_size --> 64
dropout --> keep probability = 1.0
Batch normalization
RMSProp for the VAE modules and the discriminator. Not adapting learning rates.
resize convolutions --> should I use transpose 2d Convolutions instead
latent dimension --> 2048
60000 train images (CelebA) Not using full 200k at this point.
encoder 64, 128, 256 all relu and batchnorm, + fully connected for sigma sq and mean z, no batchnorm after fully connected
decoder fully connected (88256) + reshape + batchnorm + relu , conv layers 256, 128, 64, with batch norm and relu. fully connected as final layer with tanh.

I normalize the input images before the encoder to [-1, 1] and renormalize them after the tanh in the decoder to [0,1]

Discriminator conv layers: 32, 128, 256, 256, with batch norm and relu followed by a fully connected (512) relu layer with batch norm and finally a fully connected with dimension 1 and a sigmoid.

I have let this run for about 55 epochs and my reconstructed images look like this:

CelebA-reconstruction.pdf

Does this setup seem right to you? When did you start getting more life like results?

kind regards,

Riaan