Questions about training image-based detectors on AV2 dataset

[tomztyang]

Hi,

I am wondering if it is possible to train image-based methods on the AV2 sensor dataset.

I find that the timestamps between LiDAR data and Ring-image data are not aligned. I tried to project the annotations on the corresponding ring images and found that they were not aligned well. So I am wondering whether the AV2 sensor dataset currently is only valid for detectors using LiDAR point cloud inputs but not for image-based detectors?

[benjaminrwilson]

Hi @tomztyang, thanks for your interest.

Would you mind posting some examples with the code you're using to project the cuboids?

Ben

[tomztyang]

Hi @benjaminrwilson , thanks for the help!

I align the camera and lidar timestamp according to the "_build_synchronization_cache" (av2.datasets.sensor.sensor_dataloader._build_synchronization_cache).

• However, The timestamps of LiDAR and Camera are still not aligned. For example, for the LiDAR scene with log_id (00a6ffc1-6ce9-3bc3-a060-6006e9893a1a) and timestamp (315967376959702000), the aligned timestamp for ring-cameras are:
  * ring_front_center: 315967376949927221
  * ring_front_left: 315967376937425442
  * ring_front_right: 315967376962451246
  * ring_rear_left: 315967376957428273
  * ring_rear_right: 315967376942441185
  * ring_side_left: 315967376972412936
  * ring_side_right: 315967376977482493

Then, I project the GT 3D annotations on timestamp (315967376959702000) to the ring-front-left camera for instance and show the projected results at below:

As you can see here, there is a misalignment between 3d GT and camera.

Therefore, I am wondering can you reproduce this case?

Regards,
Zetong

[Syedaliraza005]

hi i am having issues in running av2.torch.data_loaders.detection.py file to run object detection av2-api-main\tutorials\3d_object_detection file please help to run it