How to improve performance on ScanNet scenes?

[yashbhalgat]

Hi @sunset1995, thank you for this work! The code is very clear and useful.

I am trying to train the model on a scene (scene0000_00) from ScanNet dataset. I got the model to train correctly, but the rendering results are slightly poor, for example (top left: ground truth, top right: rendered image, bottom right: rendered depth):

The train PSNR reaches a high value of 25, but test PSNR is only 13. Also, the rendered images and depths look pretty good on the training images, but fail on testset image as shown image.

I am wondering what can be done to improve the performance. I tried Instant-NGP with similar near_far parameters (i.e. [0.1, 10.0]) and the results were decent. Can you please advise?