ROS publish rate is significantly below camera framerate

[rgov]

System Description
Camera Name and Model: FLIR Blackfly S BFS-U3-50S5C-C
Operating System: Ubuntu 18.04
Spinnaker Version: 2.4.0.143 (arm64)
ROS Version: Melodic

Computer details
Processor: Jetson TX2 (4 ARMv8 cores)
RAM: 8 GB

Describe the bug
A clear and concise description of what the bug is.

My camera is capable of 20 FPS. I confirmed this by modifying the Acquisition.cpp example file from the Spinnaker SDK to capture 200 frames (without saving them to the disk) and the timing worked out to precisely 20 FPS.

However, when I run either node_acquisition.launch or acquisition.launch, configured to send to ROS but not save to disk, the speed is about 10.2 FPS, almost half.

[rgov]

Attaching a callgrind trace: callgrind.listener.19483.zip

[ghost]

@rgov one issue i have often seen is when the actual exposure time is longer than required for frame rate. You can quickly test by setting the exposure to something like 2000 us.

Additionally try lowering the required datarate by reducing resolution by setting binning to 2 and the color to grayscale. This will help understand where the latency comes from.

[rgov]

With exposure_time set to 2000 (µs, seems undocumented?) I get a modest increase, to 12.4 FPS. But the Spinnaker Acquisition.cpp demo does not set exposure time and seems to get 20 FPS.

[vik748]

@rgov this could be due to the overhead associated with ROS's camera transport. 5MP is quite a bit of data for serializing / deserializing. Do you get any improvement with grayscale (1/3 reduction in data) and setting binning to 2 (1/4 reduction in data) ?

[vik748]

Also, we have seen issues with newer version of spinnaker, can you try with spinnaker 2.2.0.48

[vik748]

@rgov any updates on your issue?

[rgov]

It is not resolved yet but I have been working on other parts of my system. I'd like to get to the bottom of it.

[nevangeliouNYUAD]

A personal view on this: I think the soft triggering methodology creates some sort of overhead.

The Acquisition.cpp from FLIR achieves on a GigE Blackfly-S 35Hz in color mode on an i7 Intel NUC (Ubuntu 18, Spinnaker SDK 2.0.0.48). I added the image conversion and ROS publishing and rate reduces to 32Hz. If I use this package with same camera settings, best rate I could get was 18Hz. Exposure time@5000us.

For users who want high rates you can also have a look at the FLIR SDK, convert the image with padding and publish to ros with cv_bridge/image_transport. Not all packages can be super-efficient at all configs. This package is definitely a life-saver w.r.t. master triggering though.

With thanks to the authors.

[vik748]

@nevangeliouNYUAD There is also another existing ROS driver https://github.com/ros-drivers/flir_camera_driver which might work better for some.