Skip to content

3D normal map estimation pipeline using low resolution and highly occluded images

Notifications You must be signed in to change notification settings

VigneshS10/Tri-Stage-Occlusion-Handling-Normal-map-Estimation-Algorithm

Repository files navigation

TRI-STAGE OCCLUSION HANDLING NORMAL MAP ESTIMATION ALGORITHM (TSOHNMEA)

SAMSUNG PRISM PROGRAM

architecture


This repository is the official implementation of the TSOHNMEA. The project report can be viewed using this link. Complete end-to-end pipeline code is not included yet.

Requirements

To install requirements:

pip install -r requirements.txt

Output

As the end-to-end pipeline code is not included, users must manually take the output from each .ipynb section. The order to extracts and input the outputs is

  • background_occlusion_handler.ipynb (not included yet, so we suggest to use an image without any background to test the rest of the pipeline)
  • Real_ESRGAN_blur_occlusion.ipynb
  • PiFUHD_normal_map_estimation.ipynb

Results

  • ORDINARY NORMAL MAP ESTIMATION USING PIFU-HD MODEL:
PIFU-HD.mp4
  • OUR TRI-STAGE OCCLUSION HANDLING NORMAL MAP ESTIMATION PIPELINE:

Background, Shadow and Blur occlusion of the original input image is handled and the preprocessed input image is sent to the normal map estimation model

Our-pipeline.mp4

TODO

  • Upload complete end-to-end pipeline code (not included right now because further improvements are being made).
  • Include background occlusion handler code (not included right now because more advanced and better architectures are being explored).
  • Handle object occlusion.

References

https://github.com/tensorflow/models/tree/master/research/deeplab
https://github.com/xinntao/Real-ESRGAN
https://github.com/facebookresearch/pifuhd

Contact

For any queries, feel free to contact at vignesh.nitt10@gmail.com.

Releases

No releases published

Packages

No packages published