NeRF Relighting & Compositing

NeRF-RC builds on the existing literature in 3D graphics and computer vision to develop a solution that integrates 3D mesh objects into a relightable scene generated by different NeRF (Neural Radiance Field) architectures. The solution allows for adjusting global lighting in a way that affects both the NeRF scene and the embedded 3D meshes. Integrating 3D meshes into NeRF scenes will create more realistic and dynamic environments. The ability to adjust global lighting will further enhance the realism of the overall scene. NeRF and NeRF Relighting have gained significant attention in both the computer vision and computer graphics communities in recent years due to their impressive results in reconstructing and relighting 3D scenes.

Completed under the supervision of Prof. Dr. Janne Heikkilä and Phong Nguyen

Thesis

http://jultika.oulu.fi/Record/nbnfioulu-202306152524

Pipeline

1. NeRF-OSR to extract scene illumination.
2. NVDiffRec to extract object shape, BRDF, and illumination.
3. Combine novel lighting + novel viewpoint.
4. Render the result.

1. Scene extraction

2. Object mesh extraction

3. Scene-Object compositing

Additional

NeRF Multi-Layer Perceptron (MLP)