Skip to content

CVI-SZU/GazeFlow

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

20 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GazeFlow

The offical implementation of our paper GazeFlow: Gaze Redirection with Normalizing Flows. pdf


Overview

  • model.py: GazeFlow model.
  • train.py: Trainer for GazeFlow.
  • evaluation.py: Containing methods encode and decode for gaze redirection.
  • imgs: Images for testing. Accessing more images for testing can be found in GazeFlowDemo.ipynb.
  • GazeFlowDemo.ipynb: A pre-trained GazeFlow demo for testing. DEMO is now available.

Package requirements

Our codes are based on tensorflow 2.3 and open source normalizing flows package TFGENZOO. Codes should work fine on tensorflow ≥ 2.3. You can easily build your environment by pip install -r requirements.txt.


Data

MPIIGaze (low resolution)

ETH-XGaze (high quality)

UT-MultiView (gray-scale)

Preprocess

Please check the data/data_preprocess.py, it provides a simple scripts for processing the MPIIGaze dataset.

If you want to train your own dataset, just keep the input format like this:

{
    'right_eye': 
        {'img': right_eye_img, 'headpose': right_headpose, 'gaze': right_gaze},
    'left_eye': 
        {'img': left_eye_img, 'headpose': left_headpose, 'gaze': left_gaze},
}

Our preprocessed dataset

  1. MPIIGaze dataset is uploaded to GoogleDrive.

  2. The other processed datasets (UT, ETH-XGaze) are lost. But for those full face images, our methods just need to crop and normalize the eye image from the face image. You can fellow the step in data/data_preprocess.py to process your own dataset.


Train

CUDA_VISIBLE_DEVICES=0 python train.py --images-width=64 --images-height=32 --K=18 --L=3 --datapath=/your_path_to_preprocess_data/xgaze_64x32.tfrecords

Test and pretrained model

We provided a ETH-XGaze pre-trained GazeFlow demo for gaze redirection. Check out GazeFlow Colab Demo. Pre-trained model will be downloaded by running the notebook. For convenience, you can upload it to Colab and run it.

CUDA_VISIBLE_DEVICES=0 python train.py --BATCH-SIZE=32 --images-width=128 --images-height=128 --K=18 --L=5 --condition-shape=4 --total-take=34000 --datapath=/path_to_your_preprocessed_data/mpiiface.tfrecords

More Images

Samples

  1. ETH-XGaze Samples

  1. MPII-Gaze Samples

  1. UT-MultiView Samples

  1. Gaze Correction, Crop eyes image and paste back.

  1. Full face gaze redirect, crop eyes and paste with poisson blending


Pretrained Models

  1. Google Drive: ETH-XGaze

Overview

  • model.py: GazeFlow model.
  • train.py: Trainer for GazeFlow.
  • evaluation.py: Containing methods encode and decode for gaze redirection.
  • imgs: Images for testing. Accessing more images for testing can be found in GazeFlowDemo.ipynb.
  • GazeFlowDemo.ipynb: A pre-trained GazeFlow demo for testing. DEMO is now available.

Package requirements

Our codes are based on tensorflow 2.3 and open source normalizing flows package TFGENZOO. Codes should work fine on tensorflow ≥ 2.3. You can easily build your environment by pip install -r requirements.txt.

Data

MPIIGaze (low resolution)

ETH-XGaze (high quality)

UT-MultiView (gray-scale)

Preprocess

Please check the data/data_preprocess.py, it provides a simple scripts for processing the MPIIGaze dataset.

If you want to train your own dataset, just keep the input format like this:

{
    'right_eye': 
        {'img': right_eye_img, 'headpose': right_headpose, 'gaze': right_gaze},
    'left_eye': 
        {'img': left_eye_img, 'headpose': left_headpose, 'gaze': left_gaze},
}

Train

CUDA_VISIBLE_DEVICES=0 python train.py --images-width=64 --images-height=32 --K=18 --L=3 --datapath=/your_path_to_preprocess_data/xgaze_64x32.tfrecords

Test and pretrained model

We provided a ETH-XGaze pre-trained GazeFlow demo for gaze redirection. Check out GazeFlow Colab Demo. Pre-trained model will be downloaded by running the notebook. For convenience, you can upload it to Colab and run it.

CUDA_VISIBLE_DEVICES=0 python train.py --BATCH-SIZE=32 --images-width=128 --images-height=128 --K=18 --L=5 --condition-shape=4 --total-take=34000 --datapath=/path_to_your_preprocessed_data/mpiiface.tfrecords

More Image

Samples

  1. ETH-XGaze Samples

  1. MPII-Gaze Samples

  1. UT-MultiView Samples

  1. Gaze Correction, Crop eyes image and paste back.

  1. Full face gaze redirect, crop eyes and paste with poisson blending

Pretrained Models

  1. ETH-XGaze-Redirection

Reference

  1. TFGENZOO

Releases

No releases published

Packages

No packages published

Languages