ASSERTION FAILED Error while launch

[ajay1606]

Hello,

System Description
Camera Name and Model: Blackfly S BFS-PGE-23S3C
Operating System: Ubuntu 18.04
Spinnaker Version: spinnaker-2.4.0.143-Ubuntu18.04
ROS Version: Melodic

Computer details
Processor: Intel® Core™ i7-5820K CPU @ 3.30GHz × 12
RAM: 32 GB

For Ethernet cameras:
Currently connected with Ethernet interface directly to host machine

Do the cameras work with SpinView?
Yes, It does work well with Spin view

Describe the bug
Camera id replaced in the test_params file and tried to launch ROS package using
roslaunch spinnaker_sdk_camera_driver acquisition.launch, But getting error message like below.

Error Messages
SUMMARY

PARAMETERS

• /acquisition_node/binning: 1
• /acquisition_node/cam_aliases: ['cam0']
• /acquisition_node/cam_ids: [18486426]
• /acquisition_node/color: False
• /acquisition_node/delay: 1.0
• /acquisition_node/distortion_coeffs: [[-0.021141875266...
• /acquisition_node/distortion_model: plumb_bob
• /acquisition_node/exposure_time: 0
• /acquisition_node/external_trigger: False
• /acquisition_node/flip_horizontal: [False]
• /acquisition_node/flip_vertical: [False]
• /acquisition_node/frames: 3400
• /acquisition_node/gain: 0
• /acquisition_node/image_height: 1080
• /acquisition_node/image_width: 1440
• /acquisition_node/intrinsic_coeffs: [[1886.9232141485...
• /acquisition_node/live: False
• /acquisition_node/live_grid: False
• /acquisition_node/master_cam: 18486426
• /acquisition_node/max_rate_save: False
• /acquisition_node/projection_coeffs: [[913.700317, 0.0...
• /acquisition_node/rectification_coeffs: [[1.0, 0.0, 0.0, ...
• /acquisition_node/region_of_interest/height: 0
• /acquisition_node/region_of_interest/width: 0
• /acquisition_node/region_of_interest/x_offset: 0
• /acquisition_node/region_of_interest/y_offset: 0
• /acquisition_node/save: False
• /acquisition_node/save_path: ~
• /acquisition_node/save_type: bmp
• /acquisition_node/skip: 20
• /acquisition_node/soft_framerate: 30
• /acquisition_node/target_grey_value: 0
• /acquisition_node/tf_prefix:
• /acquisition_node/time: False
• /acquisition_node/to_ros: True
• /acquisition_node/utstamps: False
• /rosdistro: melodic
• /rosversion: 1.14.11

NODES
/
acquisition_node (nodelet/nodelet)
vision_nodelet_manager (nodelet/nodelet)

auto-starting new master
process[master]: started with pid [12692]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 8161f1f0-d595-11eb-a330-3497f62ae04e
process[rosout-1]: started with pid [12703]
started core service [/rosout]
process[vision_nodelet_manager-2]: started with pid [12710]
process[acquisition_node-3]: started with pid [12711]
[ INFO] [1624612367.121071209]: Initializing nodelet with 12 worker threads.
[ INFO] [1624612367.235829001]: Initializing nodelet
[ INFO] [1624612367.236720468]: [ OK ] USB memory: 1000 MB
[ INFO] [1624612367.236773716]: *** PARAMETER SETTINGS ***
[ INFO] [1624612367.236788854]: ** Date = 20210625
[ INFO] [1624612367.237603734]: Save path set via parameter to: /home/ajay
[ INFO] [1624612367.237640549]: Camera IDs:
[ INFO] [1624612367.238216752]: 18486426
[ INFO] [1624612367.238789125]: Camera Aliases:
[ INFO] [1624612367.238826657]: 18486426 >> cam0
[ INFO] [1624612367.239358292]: External trigger: false
[ INFO] [1624612367.240385761]: Unique time stamps for each camera: false
[ INFO] [1624612367.240908935]: color set to: false
[ INFO] [1624612367.241461724]: 18486426 flip_horizontal 0
[ INFO] [1624612367.241977402]: 18486426 flip_vertical 0
[ INFO] [1624612367.242483890]: Exporting images to ROS: true
[ INFO] [1624612367.242994113]: Showing live images setting: false
[ INFO] [1624612367.243504674]: Showing grid-style live images setting: false
[ INFO] [1624612367.244018402]: Max Rate Save Mode: false
[ INFO] [1624612367.244266621]: Displaying timing details: false
[ INFO] [1624612367.244524327]: No. of images to skip set to: 20
[ INFO] [1624612367.244777578]: Init sleep delays set to : 1.00 sec
[ INFO] [1624612367.245012727]: 'exposure_time'=0, Setting autoexposure
[ INFO] [1624612367.245242641]: 'gain' Parameter was zero or negative, using Auto gain based on target grey value
[ INFO] [1624612367.245478504]: 'target_grey_value'=0, Setting AutoExposureTargetGreyValueAuto to Continuous/ auto
[ INFO] [1624612367.245707548]: Binning set to: 1
[ INFO] [1624612367.245937309]: Using Software rate control, rate set to: 30
[ INFO] [1624612367.246164036]: Saving images set to: 0
[ INFO] [1624612367.246843124]: tf_prefix set to:
[ INFO] [1624612367.248076987]: Region of Interest set to width: 0 height: 0 offset_x: 0 offset_y: 0
[ INFO] [1624612367.248414040]: Camera Intrinsic Paramters:
[ INFO] [1624612367.248699774]: 1886.923214 0.000000 604.721488 0.000000 1886.666877 493.477267 0.000000 0.000000 1.000000
[ INFO] [1624612367.249024244]: Camera Distortion Paramters:
[ INFO] [1624612367.249082624]: -0.021142 -0.373387 2.385983 3.282457
[ INFO] [1624612367.249393087]: Camera Rectification Paramters:
[ INFO] [1624612367.249456960]: 1.000000 0.000000 0.000000 0.000000 1.000000 0.000000 0.000000 0.000000 1.000000
[ INFO] [1624612367.249763423]: Camera Projection Paramters:
[ INFO] [1624612367.249820635]: 913.700317 0.000000 953.448302 0.000000 0.000000 1063.296631 777.871993 0.000000 0.000000 0.000000 1.000000 0.000000
[ INFO] [1624612367.249840551]: Camera coeffs provided, camera info messges will be published.
[ INFO] [1624612367.249895965]: *** SYSTEM INFORMATION ***
[ INFO] [1624612367.249916695]: Creating system instance...
[ INFO] [1624612367.249931206]: spinnaker_sdk_camera_driver package version: 1.1.1
[ INFO] [1624612368.094326697]: Spinnaker library version: 2.4.0.143
[ INFO] [1624612368.094399070]: Retreiving list of cameras...
[ INFO] [1624612368.099277513]: Numer of cameras found: 1
[ INFO] [1624612368.099341686]: Cameras connected: 1
[ INFO] [1624612368.100866583]: -18486426 Blackfly S BFS-PGE-23S3C 1807.0.66.0
[ INFO] [1624612368.146744431]: Dynamic Reconfigure: Level : 4294967295
[ INFO] [1624612368.147779554]: *** FLUSH SEQUENCE ***
[ INFO] [1624612368.147822868]: Initializing cameras...
[ INFO] [1624612369.382937776]: Deinitializing cameras...
[ INFO] [1624612369.588629055]: All cameras deinitialized.
[ INFO] [1624612371.588816715]: Initializing cameras...
[ INFO] [1624612372.715657274]: onInit Initialized
[ INFO] [1624612372.715799753]: *** ACQUISITION ***
[FATAL] [1624612372.774290124]: ASSERTION FAILED
file = /home/ajay/spinnaker_ws/src/spinnaker_camera_driver/src/camera.cpp
line = 59
cond = frameID_ == 0
message =
[FATAL] [1624612372.774372952]: First frame ID was not zero! Might cause sync issues later...
[FATAL] [1624612372.774400020]:

[vision_nodelet_manager-2] process has died [pid 12710, exit code -5, cmd /opt/ros/melodic/lib/nodelet/nodelet manager __name:=vision_nodelet_manager __log:=/home/ajay/.ros/log/8161f1f0-d595-11eb-a330-3497f62ae04e/vision_nodelet_manager-2.log].

Screenshots

Regards,
Ajay

[vik748]

Pls fill all the info in the bug report template. You can try to comment that assertion line.

[ajay1606]

@vik748 Thanks for your quick response and here with filled the bug report as you mention.
Could you please help to fix this issue?

Thanks,
Ajay

[vik748]

If you see our readme, we have known compatibility issues with spinnaker versions after 2.2.0.48. I'd suggest trying with that first. Also, you can try commenting the line that checks for the frame number at

spinnaker_sdk_camera_driver/src/camera.cpp

Line 59 in 9c96330

ROS_ASSERT_MSG(frameID_ == 0 ,"First frame ID was not zero! Might cause sync issues later...");

Unfortunately, I don't have access to an ethernet camera so can't really test it out.

[ajay1606]

@vik748 Thank you for the suggestions!

Regarding compatibility issues with spinnaker versions as mentioned in the README file;

1. Known Issues

Where can I find the known fixes?

2. Previous releases
As mentioned in the README, Spinnaker 2.2.0.48 has been tested with Ubuntu18.04, So where can I find previous versions to download. Because in the official link https://www.flir.com/products/spinnaker-sdk/ I can only find the recent Spinnaker version.
Please share the download link to spinnaker version 2.2.0.48 for Ubuntu 18.04 (AMD64)

3. Line comment
I have tried commenting the line as you suggested
spinnaker_sdk_camera_driver/src/camera.cpp (Line 59)
ROS_ASSERT_MSG(frameID_ == 0 ,"First frame ID was not zero! Might cause sync issues later...");
But ROS node terminates with following error

4. Spinnaker View
Also, Spin View also not streaming any data, I can only see a blank screen in Spin View.

Sorry that I can't figure out the issues, looking for your kind assistance.

Regards,
Ajay

[ghost]

https://coe.northeastern.edu/fieldrobotics/spinnaker_sdk_archive/spinnaker-2.2.0.48-amd64-pkg.tar.gz

Before you use the ROS driver, you have to make sure that the camera works with SpinView. If you have any issues with that part, FLIR can help.

[ajay1606]

@vik748 Thank you.
Currently, i have a spinnaker version to 2.2.0.48, and able to visualize data SpinView.

Also, After launch roslaunch spinnaker_sdk_camera_driver acquisition.launch, there is no error message.
But once i start rqt_image_view or rviz, Node terminates with an error message.

Changed Spinnaker Version:

Error Message:

Regards,
Ajay

[vik748]

@ajay1606 can you share the params.yaml file that you are using.

[ajay1606]

@vik748 here with sharing test_params.yaml file currently in use, intrinsic params in the file are default one.

cam_ids:
- 18486426
cam_aliases:
- cam0
master_cam: 18486426
skip: 20
delay: 1.0

# Assign all the follwing via launch file to prevent confusion and conflict

#save_path: ~/projects/data
#save_type: .bmp #binary or .tiff or .bmp
#binning: 1 # going from 2 to 1 requires cameras to be unplugged and replugged
#color: false
#frames: 50
#soft_framerate: 20 # this frame rate reflects to the software frame rate set using ros::rate
#exp: 997
#to_ros: true  #When to_ros is not selected, but live is selected, pressing 'space' exports single image to ROS

#Camera info message details
distortion_model: plumb_bob
image_height: 1080
image_width: 1440
distortion_coeffs:
- [-0.021141875266089592, -0.3733872931278025, 2.385982550579459, 3.2824571732099725]


#specified as [fx  0 cx 0 fy cy 0  0  1]
intrinsic_coeffs:
- [1886.9232141485886, 0.0, 604.7214878709341, 0.0, 1886.6668765711668, 493.47726714719823, 0.0, 0.0, 1.0]

rectification_coeffs:
- [1.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000]

projection_coeffs:
- [913.700317, 0.000000, 953.448302, 0.000000, 0.000000, 1063.296631, 777.871993, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000]

flip_horizontal:
- false

flip_vertical:
- false

region_of_interest:
  width: 0
  height: 0
  x_offset: 0
  y_offset: 0

[vik748]

Without a camera on hand I am not sure what could be going on. I suggest trying to reduce the bandwidth by using a low FPS, set binning to 2 to reduce resolution and using grayscale.

[ajay1606]

@vik748 I have tried with the following configuration settings in params.yaml file:
binning: 2 (before 1)
frames: 30 (# before 50)
#soft_framerate: 20
Streaming works well, until following messages appear

[ WARN] [1624845702.362489943]: Frames are being skipped!
[ WARN] [1624845759.880872530]: Image incomplete with image status 3!

Once these messages appeared, the ROS node terminates automatically with the following error:

[FATAL] [1624845761.896066994]: Spinnaker: Failed waiting for EventData on NEW_BUFFER_DATA event. [-1011]
 Likely reason is that slaves are not triggered. Check GPIO cables

[vision_nodelet_manager-1] process has died [pid 18563, exit code -11, cmd /opt/ros/melodic/lib/nodelet/nodelet manager __name:=vision_nodelet_manager __log:=/home/ajay/.ros/log/43a81cd4-d745-11eb-939b-3497f62ae04e/vision_nodelet_manager-1.log].

Also, When I looked at the acquisition. launch file default parameters look like this!

 **<arg name="frames"            default="3400"   doc="Numer of frames to save/view 0=ON"/>**
  <arg name="live"              default="false"  doc="Show images on screen GUI"/>
  <arg name="live_grid"         default="false"  doc="Show images on screen GUI in a grid"/>
  <arg name="output"            default="screen" doc="display output to screen or log file"/>
  <arg name="save"              default="false"  doc="flag whether images should be saved or not"/>
  <arg name="save_path"         default="~"      doc="location to save the image data"/>
  <arg name="save_type"         default="bmp"    doc="Type of file type to save to when saving images locally: binary, tiff, bmp, jpeg etc." />
  <arg name="soft_framerate"    default="30"     doc="When hybrid software triggering is used, this controls the FPS, 0=as fast as possible"/>
  <arg name="time"              default="false"  doc="Show time/FPS on output"/>
  <arg name="to_ros"            default="true"   doc="Flag whether images should be published to ROS" />
  <arg name="utstamps"          default="false"  doc="Flag whether each image should have Unique timestamps vs the master cams time stamp for all" />
  <arg name="max_rate_save"     default="false"  doc="Flag for max rate mode which is when the master triggerst the slaves and saves images at maximum rate possible" />

But inside the capture.cpp parameters looks likes this;
// default values for the parameters are set here. Should be removed eventually!!

    exposure_time_ = 0 ; // default as 0 = auto exposure
    soft_framerate_ = 20; //default soft framrate
    gain_ = 0;
    ext_ = ".bmp";
    SOFT_FRAME_RATE_CTRL_ = false;
    LIVE_ = false;
    TIME_BENCHMARK_ = false;
    MASTER_TIMESTAMP_FOR_ALL_ = true;
    EXTERNAL_TRIGGER_ = false;
    EXPORT_TO_ROS_ = false;
    PUBLISH_CAM_INFO_ = false;
    SAVE_ = false;
    SAVE_BIN_ = false;
    nframes_ = -1;
    FIXED_NUM_FRAMES_ = false;
    MAX_RATE_SAVE_ = false;
    region_of_interest_set_ = false;
    skip_num_ = 20;
    init_delay_ = 1;
    binning_ = 1;
    SPINNAKER_GET_NEXT_IMAGE_TIMEOUT_ = 2000;
    todays_date_ = todays_date();

Maybe the current error message Frames are being skipped! could be because of this nframes_ parameter? Just guess!

ANd, Do we have to estimate intrinsic params manually before launch ROS node ?

Regards,
Ajay

[vik748]

Frames is when you are saving the frames it will quit after the number of frames. The launch file overrides the settings in params.yaml. so set the binning, color and soft_fps in the launch file.

Yes you have the calibrate the cameras manually, this node just publishes those values.

I think your issues are related to the network settings / network load. You should check FLIR recommendations for network configuration.

[ajay1606]

@vik748 Thanks for your continuous assistance.
I have modified the changes in the Launch file and the camera working well without any interruption.

But this warning message keep accuring !

[ WARN] [1624931876.891512994]: Frames are being skipped!
[ WARN] [1624931878.749213756]: Image incomplete with image status 3!
[ WARN] [1624931878.791487400]: Frames are being skipped!
[ WARN] [1624931878.947170760]: Image incomplete with image status 3!
[ WARN] [1624931878.991406860]: Frames are being skipped!
[ WARN] [1624931903.231906282]: Image incomplete with image status 3!
[ WARN] [1624931903.270368710]: Frames are being skipped!

Previously, whenever this message occurs, the ROS node terminated automatically, but not this time.
So is this warning message need to fix or just ignore ?

And, if the issue associated with Network setting/ Network load, then it should appear similarly in the SpinView tool also right?
But it has no issues with the SpinView tool, But only with ROS node problem occuring.

Regards,
Ajay

[vik748]

This warning pops up when the camera buffer fills up and you don't extract the images in the time, so that the sequence number shows missing ones. This is less likely to happen when using software triggering. Also if the buffer is full, you would see a lag in the images.
It seems that once in a while corrupted or incomplete images are coming through and that is why you probably see that warning. It is possible that SpinView just ignores bad images and doesn't show such messages. I still suspect this has something to do with some network configuration.

[ajay1606]

@vik748 Appreciate your suggestions and completely agree with you. I will communicate with the FLIR once regarding network configuration.

[orange2xiaoyu]

Hi, I have the same issue, could you tell me how to modify the Launch file? Only change this nframes_ parameter?

[ajay1606]

@orange2xiaoyu As mentioned by @vik748 The launch file overrides the settings in params.yaml. so set the binning, color and soft_fps in the launch file.

launch/acquisition.launch

 <!-- acquisition spinnaker params-->
  <arg name="binning"           default="2"      doc="Binning for cameras, when changing from 2 to 1 cameras need to be unplugged and replugged"/>
  <arg name="color"             default="true"  doc="Should color images be used (only works on models that support color images)"/>
 <arg name="soft_framerate"    default="20"     doc="When hybrid software triggering is used, this controls the FPS, 0=as fast as possible"/>

Just change above params.

[orange2xiaoyu]

@ajay1606 Thank you for your reply.
But when I modified the binning, color and soft_fps in the launch file as your launch file, the same error still occurred.

[ajay1606]

@orange2xiaoyu Try commenting below the line in the source spinnaker_sdk_camera_driver/src/camera.cpp and try again.

ROS_ASSERT_MSG(frameID_ == 0 ,"First frame ID was not zero! Might cause sync issues later...");

[orange2xiaoyu]

@ajay1606 Thank you very much. I comment the line in the source spinnaker_sdk_camera_driver/src/camera.cpp and try again, but the same error still occurred.

[ajay1606]

@orange2xiaoyu In my case, ASSERTION failed error messaged caused to due to line 59, in the campera.cpp and commenting that resolved my issue. Please check your error message cause, or please share the error message once here.

[acarrillo]

Hi, this is still a very reproducible issue! I have the exact same setup described by @ajay1606 ; I can see my camera stream just fine in spinview but I experience the exact same symptom as in this comment, #139 (comment), and setting binning, color, and soft_framerate does not solve my problem. @ajay1606 , I have no idea how you managed to dig yourself out of that hole!

[ajay1606]

@acarrillo I managed to overcome this issue as mentioned above. Please share the actual error message that appears in your camera. There could be another reason for that cause.