- Fix depth supervision, migrate depth estimation model to omnidata.
- Add normal supervision (also by omnidata).
image-to-3d.mp4
Improvement on mesh quality & DMTet finetuning support.
output.mp4
- adopt ideas from Fantasia3D to concatenate normal and mask as the latent code in a warm up stage, which shows faster convergence of shape.
relight.mp4
- Use an MLP to predict the surface normals as in Magic3D to avoid finite difference / second order gradient, generation quality is greatly improved.
- More efficient two-pass raymarching in training inspired by nerfacc.
out.mp4
- Support Stable-diffusion 2.0 base.
- Add the vanilla backbone that is pure-pytorch.
- The shading (partially) starts to work, at least it won't make scene empty. For some prompts, it shows better results (less severe Janus problem). The textureless rendering mode is still disabled.
- Enable shading by default (--latent_iter_ratio 1000).
- Basic reproduction finished.
- Non --cuda_ray, --tcnn are not working, need to fix.
- Shading is not working, disabled in utils.py for now. Surface normals are bad.
- Use an entropy loss to regularize weights_sum (alpha), the original L2 reg always leads to degenerated geometry...