Custom gstreamer plugins

[dakejahl]

I would be nice to be able to use the nvidia plugins (nvarguscamerasrc, nvvidconv, nvv4l2h264enc, etc) on the nvidia Jetson platforms for interfacing with mipi cameras. Another useful plugin is the gst-ros-bridge which allows using a ros2 topic as the src via rosimagesrc.

Is this something that can be easily supported and is there any other interest?

[patrickelectric]

Check: #274

[patrickelectric]

Be free to help with the development of such architecture

[joaoantoniocardoso]

Hi @dakejahl, as Patrick mentioned, we are discussing a change very much in this direction, more specifically, the third configuration of the architecture introduced by Patrick would allow the user to use arbitrary codecs, can you read and check if it makes sense? #274

Feel free to collaborate there.

---

About the gst-ros-bridge, that would be awesome to have it! We'd need to create a new source and a sink for it, which is not difficult to do. If you have an interest in coding it, I can guide you.

Thanks

[AlexKlimaj]

I am also interested in this.

[dakejahl]

I'm very noob at rust and my gstreamer experience is limited to command line usage so bear with me. Perhaps it's simpler to ask if a custom source element can be used. For example when I build/install the ros-gst-bridge I can launch a pipeline as such

gst-launch-1.0 --gst-plugin-path=install/gst_bridge/lib/gst_bridge/ rosimagesrc ros-topic="/image_raw" ! queue max-size-buffers=1 ! videoconvert ! "video/x-raw,format=I420" ! x264enc bitrate=2000 tune=zerolatency speed-preset=ultrafast ! "video/x-h264,stream-format=byte-stream" ! rtph264pay config-interval=1 pt=96 ! udpsink host=192.168.0.16 port=5600 sync=false

Is this easier to achieve than asking for complete pipeline customization? Or would this still require the architecture changes proposed in #274?

[joaoantoniocardoso]

Hi @dakejahl,

there`s an easier way to add that: we could just add another source type, similar to the v4lsrc one, but without #274 we will not be able to select the codec plugin or the codec properties, but we can add it separately as a first step, sure!

[dakejahl]

@joaoantoniocardoso okay that makes sense. So what you're saying is that in the case that a hardware encoder is available I wouldn't be able to select nvv4l2h264enc. And in the case of no hw encoder it would default select x264enc but I cannot add the bitrate=2000 tune=zerolatency speed-preset=ultrafast properties.

[joaoantoniocardoso]

@dakejahl exactly. I think we can add this ROS bridge before #274 with some static configuration as a way to play with ROS, but we should work in a configuration API so it can be actually useful, right?