modifed version of vid2vid for Speech2Video and Text2Video.
- git clone
git clone git@github.com:sibozhang/vid2vid.git
- setup env torchvision need to be 0.2.2 to be compatible with torch 0.4.1
python3 -m venv ../venv/vid2vid
source ../venv/vid2vid/bin/activate
pip install --upgrade pip
pip3 install https://download.pytorch.org/whl/cu92/torch-0.4.1-cp36-cp36m-linux_x86_64.whl
pip install torchvision==0.2.2
pip install numpy
pip install dominate requests
pip install pillow
pip install opencv-python
pip install scipy
pip install pytz
Please build 'checkpoints' folder in the current folder and put trained model in it.
VidTIMIT fadg0 (English, Female) Dropbox
百度云链接:https://pan.baidu.com/s/1L1cvqwLu_uqN2cbW-bDgdA 密码:hygt
Xuesong (Chinese, Male) Dropbox
百度云链接:https://pan.baidu.com/s/1lhYRakZLnkQ8nqMuLJt_dA 密码:40ob
- Get vid2vid working
*cd vid2vid/models/flownet2_pytorch
*export CUDA_HOME=/tools/cuda-9.2.88/
*comment “--user” in /flownet2_pytorch/install.sh //so it will install to python under venv, otherwise it install to .local
*bash install.sh
- Q: File "/mnt/scratch/sibo/vid2vid/util/util.py", line 62, in tensor2im image_numpy = image_tensor.cpu().float().numpy() RuntimeError: PyTorch was compiled without NumPy support
A: pip install torch==0.4.1.post2
Speech2Video Synthesis with 3D Skeleton Regularization and Expressive Body Poses
Miao Liao*, Sibo Zhang*, Peng Wang, Hao Zhu, Xinxin Zuo, Ruigang Yang. PDF Result Video 1 min Spotlight 10 min Presentation
@inproceedings{liao2020speech2video,
title={Speech2video synthesis with 3D skeleton regularization and expressive body poses},
author={Liao, Miao and Zhang, Sibo and Wang, Peng and Zhu, Hao and Zuo, Xinxin and Yang, Ruigang},
booktitle={Proceedings of the Asian Conference on Computer Vision},
year={2020}
}
This code is based on the vid2vid framework.