Scaling Depth / Pose for Fine-Tuning

[dd2252]

Thanks for your great work!

I'm trying to fine-tune the model on non-CO3D data, and was wondering if pose and depth were normalized to some scale. Presumably the scale of CO3D and MegaDepth is somehow normalized so the network can learn. I notice in the code after reading depth CO3D normalizes it:

depthmap = (depthmap.astype(np.float32) / 65535) * np.nan_to_num(input_metadata['maximum_depth'])

I do not see any function applied to translation scale.

Any clarification on what normalization is happening in CO3D, as well as how to apply it to other datasets would be greatly appreciated, thank you!

[yocabon]

Hi,
the normalization of the 3d points is done in the criterion. I don't think we do anything at the dataset level.

check https://github.com/naver/dust3r/blob/main/dust3r/losses.py#L177C1-L177C102

[dd2252]

Thank you!!