Skip to content

Latest commit

 

History

History
114 lines (101 loc) · 4.88 KB

INSTALL.md

File metadata and controls

114 lines (101 loc) · 4.88 KB
Raw

Set up the python environment

  • OS: Ubuntu 20.04
  • CUDA>=10.1
  • We conduct the training on a single NVIDIA A100 GPU
  • Clone this repo by git clone https://github.com/fanegg/UV-Volumes.git
  • Python>=3.8 and PyTorch>=1.7 (installation via anaconda is recommended) 
    conda create -n uvvolumes python=3.8
conda activate uvvolumes

# make sure that the pytorch cuda is consistent with the system cuda
conda install pytorch==1.7.0 torchvision==0.8.0 torchaudio==0.7.0 cudatoolkit=10.1 -c pytorch
  • Install core requirements 
    pip install -r requirements.txt

cd uvvolumes
git clone https://github.com/traveller59/spconv --recursive
cd spconv
git checkout abf0acf30f5526ea93e687e3f424f62d9cd8313a
git submodule update --init --recursive
export CUDA_HOME="/usr/local/cuda-10.1"
python setup.py bdist_wheel
cd dist
pip install spconv-1.2.1-cp38-cp38-linux_x86_64.whl

Set up datasets

ZJU-Mocap dataset

  1. Download the ZJU-Mocap dataset here.
  2. Create a soft link: 
    ROOT=/path/to/uvvolumes
cd $ROOT/data
ln -s /path/to/zju_mocap zju_mocap

CMU Panoptic dataset

  1. Download the CMU Panoptic dataset here.
  2. Process the CMU Panoptic dataset for the binary mask using BackgroundMattingV2 and for SMPL parameters using EasyMocap.
  3. Create a soft link: 
    ROOT=/path/to/uvvolumes
cd $ROOT/data
ln -s /path/to/cmu_panoptic cmu_panoptic

H36M dataset

  1. Download the H36M dataset here.
  2. Process the H36M dataset for the binary mask using CIHP_PGN and for SMPL parameters using EasyMocap.
  3. Create a soft link: 
    ROOT=/path/to/uvvolumes
cd $ROOT/data
ln -s /path/to/h36m h36m

Get the pre-defined UV unwrap in DensePose

🧙 As DensePose sometimes can not estimate UV unwrap at some viewpoints or human poses, we ignore the semantic and UV-metric loss for the images without UV unwraps to set up a fault-tolerant mechanism. Given the noisy and even missing UV unwraps, we achieve inter-frame and inter-view generalization of UV renderings using RGB loss as the intrinsic constraint.

  1. Run inference on a directory of image files with the pre-trained DensePose models for the pre-defined UV unwrap.
  2. Move the directory of Densepose output IUV image files to the root path of the dataset.
  3. Organize the dataset as the following structure (the tutorial is recommended). For clarification, we also provide the example data directory. You can download it from here. Just FYI. 
    ├── /path/to/dataset
│   ├── annots.npy  // Store the camera parameters and image paths.
│   ├── params
│   │   ├── 0.npy
│   │   ├── ...
│   │   ├── 1234.npy
│   ├── vertices
│   │   ├── 0.npy
│   │   ├── ...
│   │   ├── 1234.npy
│   ├── Camera_B1  // Store the images. No restrictions on the directory name.
│   │   ├── 00000.jpg
│   │   ├── ...
│   ├── Camera_B2
│   │   ├── 00000.jpg
│   │   ├── ...
│   ├── ...
│   ├── Camera_B23
│   │   ├── 00000.jpg
│   │   ├── ...
│   ├── mask_cihp  // Store the binary mask. The directory name must be the same as the mask directory name in the config file.
│   │   ├── Camera_B1
│   │   │   ├── 00000.png
│   │   │   ├── ...
│   │   ├── Camera_B2
│   │   │   ├── 00000.png
│   │   │   ├── ...
│   │   ├── ...
│   │   ├── Camera_B23
│   │   │   ├── 00000.png
│   │   │   ├── ...
│   ├── densepose  // Store the pre-defined UV unwrap. The directory name must be the same as the densepose directory name in the config file.
│   │   ├── Camera_B1
│   │   │   ├── 00000_IUV.png
│   │   │   ├── ...
│   │   ├── Camera_B2
│   │   │   ├── 00000_IUV.png
│   │   │   ├── ...
│   │   ├── ...
│   │   ├── Camera_B23
│   │   │   ├── 00000_IUV.png
│   │   │   ├── ...