"Visual tracking is lost" in tutorial

[Taeyeon0206]

hi,
I was going through the Isaac ROS Vslam tutorial with Isaac Sim.
https://nvidia-isaac-ros.github.io/concepts/visual_slam/cuvslam/tutorial_isaac_sim.html

However, as soon as I launched isaac_ros_visual_slam, "visual tracking is lost" error kept on popping.
I tried echoing the topics published by visual_slam_node, but none of the topics published anything.
I looked at the image_raw topic published by Isaac Sim through rviz2 and it was just fine.

this is the error message printed.

What could be the problem?

By the way, I am using Ubuntu22.04 and ROS2 humble. I am running both Isaac Sim and Isaac ROS in a separate docker container. I am running Isaac Sim by run headless mode with streaming client. I followed exactly the document "development environment setup" and "isaac ROS Vslam quickstart" and installed isaac_ros_common and isaac_ros_visual_slam in the process.
Also, a strange point is, I tried Isaac ROS map localization and it worked.

[jaiveersinghNV]

Hi @Taeyeon0206 ,

Have you already gone through our recommended Troubleshooting steps for Isaac ROS Visual SLAM? The "Visual Tracking is Lost" message is a catch-all error that can have many different causes.

Were you able to successfully run the Isaac ROS Visual SLAM Quickstart using the pre-recorded data? If the Quickstart works, then we can isolate the issue to the input data coming from Isaac Sim.

[Taeyeon0206]

Hi @jaiveersinghNV
I successfully ran the Quickstart. It worked just fine.

1. Fast motion causing the motion blur in the frames
2. Low-lighting conditions.
3. The wrong camerainfo is being published.

These 3 suggestions were in the recommended troubleshooting step.
The error occurred from the beginning and without any movement, so I thought motion blur is not the case.
Since it is example environment, I thought low-lighting is not the problem either.
Also, I checked the published topic(left and right camera) using rviz2 and it showed up just fine.

[Taeyeon0206]

maybe this might help.
These are the camera info of left and right camera of front hawk.
I got the information by echoing the published topic
left:

right:

This is the exact error message that i have been receiving.

[Taeyeon0206]

@jaiveersinghNV
I changed the distortion model into pinhole model after bringing the simulation example environment in Isaac Sim and it worked.
I wonder why the camera distortion model has to be pinhole model. Looking at the visual_slam_impl.cpp code, it seems to me that it has to work in the rational_polynomial distortion model as well.

[swapnesh-wani-nvidia]

@Taeyeon0206, thanks for bringing this up. Can you verify if the images coming from Isaac Sim's setup are rectified and undistorted or not? If they are rectified and undistorted then it has to be a pinhole camera model.