object ids in tracked_objects skipped a number?

[utility-aagrawal]

Hi,

I am using norfair to track faces. I noticed that sometimes tracked_objects were skipping a number for object ids. For example, I have 3 faces and object ids are 1, 3, 4 instead of 1,2,3. What could be the reason? Might I be doing something wrong? Let me know if you need anything additional. Thanks!

[utility-aagrawal]

This is the video you can use for testing:
https://github.com/tryolabs/norfair/assets/140737044/aedd01ff-e2e3-458f-b8ba-c7be153cc038

Please refer to issue #307 for code. The only changes I have made are:

1. skip_period = 3,
2. initialization_delay = 9
3. hit_counter_max = 15

Thanks

[utility-aagrawal]

@aguscas Could you please advise on this?

[aguscas]

Hello @utility-aagrawal , sorry for my late response, I've been busy lately. To save me some time, could you also provide the functions that you use to detect the faces? (retinaface_detections_to_norfair_detections and detect_faces)
I will make some tests with that, and try to see what can be happening

[utility-aagrawal]

Sure @aguscas!

Here are those functions:

from retinaface import RetinaFace as rf
import numpy as np
from norfair import Detection
from typing import List

-- Face detection using retinaface model for a frame
def detect_faces(frame):
face_objs = rf.detect_faces(img_path = frame)
detections = []
for _, face_details in face_objs.items():
box = face_details['facial_area']
left = box[0]
top = box[1]
width = box[2] - box[0]
height = box[3] - box[1]
conf = face_details['score']

    if conf != 0: detections.append([left, top, width, height, conf, 0])    
return np.asarray(detections)

def retinaface_to_bbox_centroid(detection_as_xywh: np.ndarray) -> np.ndarray:
return np.array(
[
detection_as_xywh[0] + (detection_as_xywh[3] / 2), detection_as_xywh[1] + (detection_as_xywh[2] / 2)
]
)

-- Converts retinaface (left, top, width, height) prediction corner coordinates [[left_top_x, left_top_y], [right_bottom_x, right_bottom_y]]
def retinaface_to_bbox_corners(detection_as_xywh: np.ndarray) -> np.ndarray:
return np.array(
[
[detection_as_xywh[0], detection_as_xywh[1]],
[detection_as_xywh[0] + detection_as_xywh[3], detection_as_xywh[1] + detection_as_xywh[2]],
]
)

-- Converts detections to the format norfair expects i.e. Detection objects
def retinaface_detections_to_norfair_detections(
retinaface_detections_as_xywh: np.ndarray, track_points: str = "centroid" # bbox or centroid
) -> List[Detection]:
"""convert detections_as_xywh to norfair detections"""
norfair_detections: List[Detection] = []

if track_points == "centroid":
    for detection_as_xywh in retinaface_detections_as_xywh:
        centroid = retinaface_to_bbox_centroid(detection_as_xywh)
        scores = np.array([detection_as_xywh[4]])
        norfair_detections.append(
            Detection(
                points=centroid,
                scores=scores,
                label=int(detection_as_xywh[5])
            )
        )
elif track_points == "bbox":
    for detection_as_xywh in retinaface_detections_as_xywh:
        bbox = retinaface_to_bbox_corners(detection_as_xywh)
        scores = np.array(
            [detection_as_xywh[4], detection_as_xywh[4]]
        )
        norfair_detections.append(
            Detection(
                points=bbox, scores=scores, label=int(detection_as_xywh[5])
            )
        )

return norfair_detections

Let me know if you have any further questions. Thanks!

[aguscas]

Alright, I made a test using the same parameters you said for all variables, but couldn't reproduce the problem you were mentioning about skipping ids. I'm still not sure what might cause you that. Here is the video created with your code.

By the way, these models run so slow that I shouldn't even call it 'running', it's more like crawling towards the output. Hope you find other models that run faster.

Also on another note, parameters seem to need a lot of tuning, lots of ID switches there.

[utility-aagrawal]

I should have mentioned it before. It doesn't happen for every video. I have seen it happening when you have more faces (5+) in the video. I'll try to find an example video and share with you.

[utility-aagrawal]

Here's one video you can try. You'll notice that object ids 26 and 29 are skipped:

London.Walk.from.Oxford.Street.to.Carnaby.Street_30secs.mp4

[aguscas]

Are you sure you haven't made any other changes to the code here apart from the values of skip_period, initialization_delay and hit_counter_max? I am asking because I just tried that video with your code and couldn't even get to ids that high. Are you sure you haven't at least changed any threshold, like the distance_threshold or any other tracking parameter?