Handheld Scanner Help

[ECM117]

Hi all,

Noob here looking for some help and advice with setting up RTAB-Map for my project. I've read many forum posts and would like some help deciphering some things as I'm new to the software side of things. I'm looking to build a handheld 3d scanner using a Kinect v2 and Jetson Xavier NX using visual odometry as the kinect doesn't have an IMU. Use case will be in an industrial setting for equipment and piping, hopefully as small as ~1" diameter but usually 2" diameter and up. My hope is that this will be a more viable option than photogrammetry for medium sized room scanning at distances of up to 15 feet, usually involving overhead piping which photogrammetry struggles with. Some questions about this:

1. Which version of Ubuntu should I be using on my Xavier NX? Compatibility with certain libraries?

2. Which libraries are required for using the kinect v2? As I understand it from other forum posts, I need to use libfreenect2. Any sources on how exactly to setup Rtabmap with this library?

3. Is OpenCV required for visual odometry? What would be the advantages of using the nonfree SIFT/SURF version? Any hardware limitations?

4. I understand I need to build my libraries with CUDA support. Is any additional setup on my Xavier needed for CUDA support, or should it work out of the box?

5. I assume standalone Rtabmap would be fine for my purposes. Any advantages ROS offers?

Feel free to let me know of any info or requirements I've missed... Thanks!

[matlabbe]

Hi,

1. You may be fine with latest jetpack. Otherwise, rtabmap works out-of-the-box with version of dependencies associated with LTS ubuntu distros (e.g., 18.04, 20.04, 22.04, ...). Well, I didn't test yet with 24.04, which will officially released this month.
2. I would recommend to build libfreenect2 with CUDA pipeline. After you install libfreenect2, when doing cmake in rtabmap, it should be able to see it. Try libreenect2 viewer (protonect) to see if you have a decent framerate.
3. OpenCV is required, rtabmap uses it a lot. Don't need to rebuild it with nonfree module. The default features should be fine for small environments.
4. rtabmap doesn't really support CUDA explicitly, but if libfreenect2 is built with cuda, it will use it like this by default.
5. For hand-held scanning with single camera like this (and built-in in rtabmap), rtabmap standalone is fine. ROS would be useful if you have a camera not supported in rtabmap standalone, want to sync multiple cameras or lidars, or use external visual odometry.

Install also g2o to get better visual odometry.

If the environment has plenty of geometry, you may also try ICP odometry instead of visual odometry: http://official-rtab-map-forum.206.s1.nabble.com/Kinect-For-Azure-L515-ICP-lighting-invariant-mapping-td7187.html