Code for SIGGRAPH ASIA 2022 paper Differentiable Rendering using RGBXY Derivatives and Optimal Transport
Clone this repo first
We suggest create a new Anaconda environment.
conda create -n DROT python=3.8
#Install PyTorch
conda install pytorch==1.11.0 torchvision==0.12.0 torchaudio==0.11.0 cudatoolkit=11.3 -c pytorch
Install geomloss
pip install pykeops
pip install geomloss
Install customized PyTorch3D
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install -c bottler nvidiacub
git clone https://github.com/jkxing/pytorch3d
cd pytorch3d
python setup.py install
Install Nvdiffrast
git clone https://github.com/NVlabs/nvdiffrast
cd nvdiffrast
pip install .
Install other tools
conda install -c pytorch ignite
pip install tensorboard,matplotlib,xmltodict,pyglm,imageio,lpips,opencv-python
if some error occurs, the following tips may help:
- GCC version should under 10
- Install correct version of nvidia-gl driver(server version may fail)
- CUDA version/Python-dev version should exactly match everywhere
Please download sample data here and extract it to data/.
We provide all 8 scenes used in evaluation part. Please refer to experiments/evaluation/*.json for detail settings.
Usage
python experiments/evaluation/optim.py [translation/rotation/...]
We provide 12 scenes shown in paper and supplemental materials collected from 3D-FRONT for furniture layout application. Please refer to experiments/furniture/*.json for detail settings.
Usage
python experiments/furniture/optim.py [1-12]
Please download SMPL model and sample texture from here and extract it to data/human/, files should include
data/human/
├── basicModel_m_lbs_10_207_0_v1.0.0.pkl
├── tex.png
├── UV_Processed.mat
└── UV_symmetry_transforms.mat
Usage
python experiments/smpl/optim.py
You can specify random seed for different initialization.
Because the face model are not free for public, we do not provide the model and optimization code. But the code are similar to other applications, so you can implement it for your own model.
Since Jitter(JRender) doesn't support methods such as multi object rendering, we only provide a core implementation and a single object translation demo in submodule DROT-JRender. Please install jittor follow jittor/readme.md and then run demo file by the following command.
cd DROT-JRender
python demo-drl.py