What are the tracked objects?

[CassielMapersive]

Describe the situation you are working on
First of all, great library you have created I am impressed and I hope you are as happy with your library as I am using it.
So i am working on a way to track cars and pedestrians and creating a csv file for them using both detectron2 demo and the yolov7 demo to test between them.

Describe what is it that you need help with
There are a lot of different colored bboxes eg this image. I wonder what the difference is between the colored ones (which i assume to be the tracked objects) and the black bboxes (which i assume to be too low confidence to be used as a tracked object)?

I would like some clarity on that and if its possible to include the black bboxes into tracked objects as well.

Additional context
This frame has picked up 4 Objects but clearly there are many more bboxes and most of them are quite useful and I would like to add it to tracked objects.

[facundo-lezama]

Hi @CassielMapersive, thanks for those words!

Can you send a snippet of the code you're using? I believe those black boxes are detections, and the colored ones are the actual tracked objects. But I can't be sure unless you send some code, and I can see how you're calling the drawing functions.

[CassielMapersive]

Thank you @facundo-lezama for the quick reply!

I barely changed anything to the demo, here is what happens for every frame. Also I am not sure if this is relevant but

DISTANCE_THRESHOLD_BBOX: float = 5
DISTANCE_THRESHOLD_CENTROID: int = 30
MAX_DISTANCE: int = 5000
video sizes are usually (7000,3500).

Also I realise that the detections tend to be more accurate to than the tracked objects. Which in theory does not make sense since norfair uses the detections. Is there a way to improve the accuracy and sensitivity of the tracked objects?

Sorry for the confusion, I am quite new with this. :)

[facundo-lezama]

Thanks for the information!

There are a lot of different colored bboxes eg this image. I wonder what the difference is between the colored ones (which i assume to be the tracked objects) and the black bboxes (which i assume to be too low confidence to be used as a tracked object)?

The black boxes are detections, the instances created by the object detector for that specific frame. The colored ones are the tracked objects that Norfair has created based on previous detections.

Also I realise that the detections tend to be more accurate to than the tracked objects. Which in theory does not make sense since norfair uses the detections. Is there a way to improve the accuracy and sensitivity of the tracked objects?

Detections are usually more accurate than tracked objects. The reason is simple: detections are outputs from the object detector model for the specific frame, while tracked objects' positions are estimated using, most commonly, a Kalman Filter based on previous positions. If an object is detected often throughout the video, Norfair should be able to track it with high accuracy. But there are also circumstances such as occlusions, low frame rate, moving cameras and other hard scenarios, where the estimations for new positions of the objects are less accurate.

[CassielMapersive]

@facundo-lezama Thank you, I understand now.

Last questions to this thread: Is there a way to integrate another tracking algo? or change the Kalman filter for another one for example? Improve the tracking in some way or use a different approach entirely like deep sort for example?

[facundo-lezama]

@CassielMapersive, I would suggest you go over some of the typical knobs that you need to tune to get the tracker to work. Our Getting Started guide has some pointers to start with.

In our experience, this is an iterative process (and not a simple one) that requires you to tune things until you're comfortable with the results.

Norfair's tracking strategy was built on top of known algorithms like SORT, but Norfair is not a library that integrates different algorithms.

[CassielMapersive]

Thank you for the help!