An Image-based Generator Architecture for Synthetic Image Refinement

This repository contains the official TensorFlow 2.x implementation, in Jupyter Notebook, of the following paper:

An Image-based Generator Architecture for Synthetic Image Refinement
Alex Nasser
https://arxiv.org/abs/2108.04957

Abstract: Proposed are alternative generator architectures for Boundary Equilibrium Generative Adversarial Networks (BEGAN), motivated by Learning from Simulated and Unsupervised Images through Adversarial Training (SimGAN). It disentangles the need for a noise-based latent space. The generator will operate mainly as a refiner network to gain a photo-realistic presentation of the given synthetic images. It also attempts to resolve the latent space's poorly understood properties by eliminating the need for noise injection and replacing it with an image-based concept. The new flexible and simple generator architecture will also give the power to control the trade-off between restrictive refinement and expressiveness ability. Contrary to other available methods, this architecture will not require a paired or unpaired dataset of real and synthetic images for the training phase. Only a relatively small set of real images would suffice.

Usage

Use your own dataset

dataset
└── real images (you would need at least 20k real-life portrait face pictures in this folder )
    ├── xxx.jpg (the name doesn't matter)
    ├── yyy.jpg
    └── ...
└── synthetic images (use this folder to store 3d or even 2d generated synthetic images for model evaluation)
    ├── xxx.jpg (the name doesn't matter)
    ├── yyy.jpg
    └── ...

Train

Run model.ipynb in Juypyter Notebook to train and evaluate the mode