How to Project Ouster_signal image into Ouster_Point_cloud.

[imdsafi09]

Hi!

I want to create a Python ROS node that takes as input the /ouster/Signal_image and /Ouster_Point cloud published by the Ouster ROS driver and projects the signal image on the point cloud. In the Ouster Python SDK, they explain how to use it with a pre-computed XYZ look-up table but I want to integrate with ROS. Could you please let me know how it could be possible?
Ouster_lidar-128
Best regards

[Samahu]

Hi @imdsafi09, are you looking for a C++ or python implementation?

[imdsafi09]

Hi @Samahu,
Thank you for your response. I would prefer Python but C++ will work equally, and if you can provide integration with ROS2 it will make my job much easier.
regards,

[Samahu]

Hi @imdsafi09, please refer to this example that I compiled to you and other users on how to consume the range image and convert into a point cloud using ros+python. For C++ you can refer to the driver code and see how the point cloud is being composed.

[imdsafi09]

Hi, @Samahu,
Thank you for your quick response. However, I want to perform a specific vision task on a signal image, such as object detection, and then project the results into a point cloud coordinate to be visible on top of the point cloud. Please guide me through how I can use your developed scripts to solve this problem.

Regards,

[Samahu]

@imdsafi09 you are asking for very specific interactions with the software which is a little beyond the scope of this GitHub issues portal.

In any case, if you are interested in processing the signal image then all you need to change from the example I shared with you is to create another subscription to the /signal_image topic and do your computer vision tasks against it. However, in order to merge the results with the point cloud you will also need the need the values from the range image at the same time. This adds a bit of complexity to the problem, still you can achieve by utilizing ROS's message filters mechanism which allows you to combine multiple topics into one callback. Then you can process both images in the same callback and then rely on the example I provided to generate the altered point cloud.

I am not sure if you are aware of ouster-yolov5-demo which provides starter materials on how to process the sensor generate output as 2D images and also update the output.

Hope this helps.

[imdsafi09]

Thank you for all the insightful suggestions. I am already following ouster-yolov5-demo
and trying to do precisely the same thing but with a "/signal_image" topic rather than a range image. However, I will look into it further, as you suggested.

Regards,