Timestamp / picture time issue

[jeangot]

I'm not saying this is a bug, but I would like to understand what is happening and thought this may be the best place to post this. There could be a race condition somewhere.
This is what is happening: whenever my camera is detecting motion, I am feeding images to Deepstack via an automation that runs every 1 second. The automation is set in blocking mode currently so that it will not run a second time before the first process has returned (which I think is whenever Deepstack responds to the API post). Based on my logs, it seems that the actual rate at which images are being fed is more like once every 8 seconds. On a recent motion event, this is what happened:

4:47:19 Start of motion event on camera

4:47:20 HA sends an image to Deepstack

4:47:28 HA sends an image to Deepstack

4:47:35 Event received from Deepstack; car object detected
{'bounding_box': {'height': 0.241, 'width': 0.373, 'y_min': 0.233, 'x_min': 0.383, 'y_max': 0.474, 'x_max': 0.756}, 'box_area': 0.09, 'centroid': {'x': 0.57, 'y': 0.354}, 'name': 'car', 'object_type': 'vehicle', 'confidence': 92.548, 'entity_id': 'image_processing.deepstack_object_driveway', 'saved_file': '/config/snapshots/driveway/deepstack_object_driveway_2021-04-20_16-47-29.png'}

This is the content of the image filename referenced above:

4:47:36 HA sends an image to Deepstack

4:47:43 Event received from Deepstack; car object detected
{'bounding_box': {'height': 0.114, 'width': 0.1, 'y_min': 0.16, 'x_min': 0.22, 'y_max': 0.274, 'x_max': 0.32}, 'box_area': 0.011, 'centroid': {'x': 0.27, 'y': 0.217}, 'name': 'car', 'object_type': 'vehicle', 'confidence': 78.134, 'entity_id': 'image_processing.deepstack_object_driveway', 'saved_file': '/config/snapshots/driveway/deepstack_object_driveway_2021-04-20_16-47-37.png'}

This is the content of the image filename referenced above:

4:47:44 HA sends an image to Deepstack

4:47:52 HA sends an image to Deepstack

4:48:00 HA sends an image to Deepstack

4:48:08 HA sends an image to Deepstack

4:48:16 HA sends an image to Deepstack

4:48:24 HA sends an image to Deepstack

4:48:32 HA sends an image to Deepstack

4:48:39 End of motion event on camera

This is what I would like to understand:

The first event that is returned has an image containing a camera timestamp of 4:47:28 with a filename of 16:47:29. Presumably this is the picture submitted by HA to Deepstack just before at 4:47:28 and it took about 7 seconds for it to be analyzed and come back. This makes sense to me.

But on the second event: an image is sent to Deepstack at 4:47:36, event comes back at 4:47:43 with an image name of 16:47:37. This makes sense too, but if you then look at the image contained in the file, it has a camera timestamp of 4:48:32 with is roughly the timestamp of the last image sent to Deepstack during the event and after the car has already left, so there is nothing to see in the image. Yet We know that Deepstack fired an event for this submission and saw the car, so it looks like the image that was analyzed by Deepstack is the correct one, but a different image was saved in the file.

This happens consistently on all of my events, where the last one or several images saved in my directory have no object on them and a much later camera timestamp, yet I know that Deepstack saw an object at the time of the filename, which is why the image was saved in the first place.

Any idea of what might be wrong?

Thanks.

Jean

[hmakmur]

I can confirm there is an issue with deepstack processing in HA. I've been noticing the same thing and can't figure out what happened. In my case, there is a movement and processing happening at May 1, 2021, 11:12:42 AM. But the image that was processed is from last_target_detection: 2021-05-01_10-38-34. This is image from previous movement. If you look at the file date, you see

-rw-r--r-- 1 root root 1149362 May 1 11:12 deepstack_object_frontdor_cam_2021-05-01_10-38-34.png
-rw-r--r-- 1 root root 1149362 May 1 11:12 deepstack_object_frontdor_cam_latest.png
drwxr-xr-x 2 root root 4096 May 1 10:38 ./

Something a bit weird. This is not always happening so It is kinda randomly happening.

[sylarevan]

Hi all. Well, I can also confirm I have the same issue here. And this makes the whole integration useless, as the saved image does not correspond at all with what has been actually analyzed by Deepstack.

I'm using Node Red for all my automations. When a movement is detected, I use the image_processing.deepstack_object_synok_jardin service to trigger a detection. This seems to be correctly done, and when a person is detected, an event is correctly triggered, e.g.:

{
    "payload":
    {
        "event_type": "deepstack.object_detected",
        "entity_id": "image_processing.deepstack_object_synok_jardin",
        "event":
        {
            "bounding_box":
            {
                "height": 0.2,
                "width": 0.08,
                "y_min": 0.5,
                "x_min": 0.886,
                "y_max": 0.7,
                "x_max": 0.966
            },
            "box_area": 0.016,
            "centroid":
            {
                "x": 0.926,
                "y": 0.6
            },
            "name": "person",
            "object_type": "person",
            "confidence": 57.227,
            "entity_id": "image_processing.deepstack_object_synok_jardin",
            "saved_file": "/media/deepstack/deepstack_object_synok_jardin_2021-07-19_21-18-43.png"
        },
        "origin": "LOCAL",
        "time_fired": "2021-07-19T19:18:43.882124+00:00",
        "context":
        {
            "id": "7b37aa7c0a906d038ab772bcecdb5d5c",
            "parent_id": null,
            "user_id": null
        }
    },
    "topic": "deepstack.object_detected",
    "event_type": "deepstack.object_detected",
    "_msgid": "31b7e254.18981e"
}

As you can see, the timestamp of the image filename and the time_fired matches. Now, if I display the corresponding image /media/deepstack/deepstack_object_synok_jardin_2021-07-19_21-18-43.png, here is what I get:

The timestamp IN the image is 07-20-2021, 11:47:23 : in the future?!
And the image does not show any person detected. If I run again a deepstack detection "by hand" on the saved image, it does not actually detects a person... so:

• the image saved on the disk does not match the actual image use by deepstack for detection,
• the image actually saved comes from ... who knows? Some cache? The future?

So at best I can trigger a message telling me that a person has been detected, but I can not use the image saved by the integration to display the situation.

So strange ...

For reference, my configuration:

image_processing:
  - platform: deepstack_object
    ip_address: 192.168.1.12
    port: 5000
    save_file_folder: /media/deepstack/ 
    save_file_format: png
    save_timestamped_file: True         # write _{date}_{time} image
    always_save_latest_file: True       # write _latest image
    scale: 1.0
    show_boxes: True
    roi_x_min: 0
    roi_x_max: 1.0
    roi_y_min: 0.28
    roi_y_max: 1.0
    targets:
      - target: person
        confidence: 30
    source:
      - entity_id: camera.synok_jardin

Deepstack is running on a jetson nano 4GB.

[danlo315]

Hello- I am seeing the same thing here. My processed filename does not match with the timestamp in the image being processed.

My caught event:

{
    "event_type": "deepstack.object_detected",
    "data": {
        "bounding_box": {
            "height": 0.362,
            "width": 0.108,
            "y_min": 0.062,
            "x_min": 0.408,
            "y_max": 0.425,
            "x_max": 0.516
        },
        "box_area": 0.039,
        "centroid": {
            "x": 0.462,
            "y": 0.243
        },
        "name": "person",
        "object_type": "person",
        "confidence": 67.461,
        "entity_id": "image_processing.deepstack_driveway",
        "saved_file": "/config/snapshots/driveway/deepstack_driveway_**2022-06-25_23-44-59-540118.png**"
    },
    "origin": "LOCAL",
    "time_fired": "2022-06-26T03:44:59.846557+00:00",
    "context": {
        "id": "01G6F1SPE6133GBJ409Y0CRYNB",
        "parent_id": null,
        "user_id": null
    }
}

I also have save_timestamped_file: True and always_save_latest_file: True