How to use it over internet with a personal website?

[Daemonised]

Hi guys,

i need to make a project about streaming a ros topic video over internet and display it on my website.
So when i use 'rosrun webrtc_ros webrtc_ros_server_node', everything works fine, but now i want to make it communicate with my online website.

The website is already done and not a problem.

So i was wondering, is it possible to connect it over internet? and how to do it?

Thanks you and great work.

[mjsobrep]

It is possible. With the current codebase it isn't straight forward.

The first thing to realize is that the server only takes incoming connections, one connection per WebRTC connection. But if you want to work over the internet, it is very very likely that you don't have a public IP and you don't want to have ports open for incoming traffic. So you need something on the robot to connect to a web server, wait for connections, and then route them through to the robot. I am currently doing that with a pretty simple node based router/proxy. Ideally this wouldn't be needed, but it was easier for me than trying to modify the c++ code. Although I may go back and do that later.

Then on the front end you need a way to provide video/audio/etc. You can start with the viewer page and viewer javascript. These will point you to the webrtc ros web library functions which you can use in your own application. I found that I needed to modify those a tad bit to be compliant with the system I use. And I end up pulling in the videos in a react video component

Now that you have the front and back ends you need to tie them together with a web server. You probably want to have ssl certs, authentication, sessions, etc. so that you make sure the robots are who they say they are, only the people you want to connect can connect, etc. That is a huge pain. I do that with a trio of services, a REST API, a robot facing websocket server, and a web facing websocket server. In my application, I launch this all from a base docker compose with some overrides for production

Finally, if you are operating in challenging network situations, you may need a turn server.

My system is neither the most efficient nor robust, but its what I have at the moment and it works quite well across the country. This is really a ton of work. WebRTC ROS could be better setup to handle it, but it would still be a ton of work. Recently @Achllle has been showing me what freedom robotics's system can do. That is pretty cool, has webRTC built in.

[Daemonised]

Thanks for the answer.

So if i understand:
-I need to run the pretty simple node based router/proxy on the robot side.
-Then, server side, use webrtc ros web library functions to make the connection between the robot and the server in WebRTC.
-And then all the system of permission of connection between the robot and the server.

Sounds really like pain but i guess the result really worth it.

Also have you tried to see the latency of the streaming video between the robot and the frontend?

Thanks you.

[mjsobrep]

Sure thing. I just want to note that a lot of the code we are looking at using here from the LilFloSystem repo is outside of webrtc_ros. I have tested it a good bit, but it isn't being used by other people...

• Yes, run that proxy thing (take a look at the code so that you can see how authentication and message packing/unpacking works).
• Those library functions are actually for the front end (runs in the browser)
• Yes, all of that
• And also the server facing the browser
• And essentially you are taking the messages from the browser wrapping them up so that you can send them to the router thing and unpacking the messages from the router to send to the browser.

Note, this is all for one user at a time connected to the robot. If you want multiple users at once, you really need a media server. Kurento comes to mind, but I have never used it.

The latency is fantastic. However, it is dependant on the compute power on both ends, the frame rate, frame size, etc. A few days ago a friend on the west coast drove around my robot on the east coast. It was crazy how smooth the connection was. I don't have numbers unfortunately. But essentially all webrtc systems will perform comparably for latency, so you should be able to get an idea by trying out https://appr.tc/ .

[Daemonised]

Ok perfect thanks a lot.
I will check this and try to make it works.

[lukelu520]

Dear webrtc_ros friends and @mjsobrep,

In our prototyping case, our ros server is also a web server and we are welcome external traffic to our assigned webrtc port . And the server has a public ip.

If so, how does webrtc_ros work for the case ? We've followed the git: webrtc-streamer to achieve (non-ros) webrtc streaming over internet. Now we are missing the integration of ros and webrtc part.

Hope to hear your expertise.

Best,
-- Luke

[mjsobrep]

@lukelu0520 if that is all true, then when you run the webrtc ros server node, I think you should be able to go to the IP address of your robot at the specified port and see a landing page to access the video feeds on your robot: http://wiki.ros.org/webrtc_ros

[lukelu520]

Thanks for your reply @mjsobrep !

Now, we are able to see the landing page which shows:

WebRTC ROS
Camera Topics
/raspicam_node/
Image Topics

We believe that our raspicam_node is now hooked up with webrtc_ros, however, we don't see the actual image or video pop out on the landing page.

Maybe we miss something and need to setup something in somewhere, e.g. a launch file. We were wondering if you have any idea.

Hope to hear your expertise.

Thanks,
-- Luke

[mjsobrep]

mmm, yeah. The raspicam node is weird, I don't remember why, but I have found it challenging to work with. Can you try either using another camera or streaming a test video, just as a place to start troubleshooting. If that works we can go from there. You can run a test video with this: https://github.com/ros-drivers/video_stream_opencv/blob/master/launch/video_file.launch

[lukelu520]

Thanks, will try video_stream_opencv as a hello-world example.

In the meantime, I was wondering if you have any API doc. For example, what is the required message type of image topic so it could be used by the webrtc_ros node. Is it sensor_msgs/Image ?

The reason raspicam is challenging to webrtc_ros might be simply because raspicam publishes compressed image to the topic of /raspicam_node/image/compressed: the message type is sensor_msgs/CompressedImage rather than sensor_msgs/Image.

Hope to hear your expertise.

Thanks,
--Luke

[lukelu520]

Thanks, will try video_stream_opencv as a hello-world example.

In the meantime, I was wondering if you have any API doc. For example, what is the required message type of image topic so it could be used by the webrtc_ros node. Is it sensor_msgs/Image ?

The reason raspicam is challenging to webrtc_ros might be simply because raspicam publishes compressed image to the topic of /raspicam_node/image/compressed. And the message type is sensor_msgs/CompressedImage (rather than sensor_msgs/Image).

Hope to hear your expertise.

Thanks,
--Luke

Follow the logic of the quote, problem solved.

By letting raspicam publishes a topic of /raspicam_node/image in a message type of sensor_msgs/Image, then webrtc_ros_server_node could see and stream it on the landing page.