RaspberryPi_WebRTC

Using v4l2 dma hardware encoder with WebRTC to reduce CPU usage on Raspberry Pi.

Note: Pi 5 does not support hardware encoder.

Architecture

Environment

• RaspberryPi 3B + Raspberry Pi Camera v1.3
• RaspberryPi OS 64bit
• clang 12+
• boringssl replace openssl

Summary

• Latency is about 0.2~0.3 seconds.
• Temperatures up to 60~65°C.
• Using the hardware DMA encoder can reduce CPU usage down to 30~35% @720p30fps.

---

How to use

Both signalr and mqtt are the options for signaling in this project.

Preparation

1. Follow SETUP_ARM64_ENV to prepare an arm64 env for compilation (Optional)
2. Follow BUILD_WEBRTC to compile libwebrtc.a
3. Choose a signaling mechanisum
   • Follow BUILD_SIGNALR_CLIENT_CPP to compile microsoft-signalr
   • Follow BUILD_MOSQUITTO to compile mosquitto
4. Install FFmpeg and the needed packages on pi

   sudo apt install ffmpeg libboost-program-options-dev libavformat-dev libavcodec-dev libavutil-dev libswscale-dev libpulse-dev libasound2-dev libx11-dev

5. Copy the nlohmann/json.hpp to /usr/local/include
6. Run the chosen signaling server
   • Create a signalr server hub on .net
   • Follow SETUP_MOSQUITTO

Compile and run

Command line	Description	Valid values
-DUSE_SIGNALR_SIGNALING	Build the project by using SignalR as signaling.	ON, OFF
-DUSE_MQTT_SIGNALING	Build the project by using MOSQUITTO as signaling.	ON, OFF
-DBUILD_TEST	Build the test codes	recorder, mqtt, v4l2_capture, v4l2_encoder, v4l2_decoder, v4l2_scaler
-DUSE_BUILT_IN_H264	Use the built-in openh264 software encoder	ON, OFF

Build on raspberry pi and it'll output a pi_webrtc file in /build.

mkdir build
cd build
cmake .. -DCMAKE_CXX_COMPILER=/usr/bin/clang++ -DUSE_MQTT_SIGNALING=ON
make -j

Run pi_webrtc to start the service.

• If use signalr as signaling.

  ./pi_webrtc --device=/dev/video0 --fps=30 --width=1280 --height=720 --v4l2_format=mjpeg --signaling_url=http://localhost:6080/SignalingServer --enable_v4l2_dma

• If use mosquitto mqtt as signaling.

  ./pi_webrtc --device=/dev/video0 --fps=30 --width=1280 --height=720 --v4l2_format=mjpeg --mqtt_host=127.0.0.1 --mqtt_port=1883 --mqtt_username=<username> --mqtt_password=<password>  --enable_v4l2_dma

• ./pi_webrtc -h to list all available args.
• --enable_v4l2_dma only apply to --v4l2_format=h264 source stream. The VP8, VP9 are available only if the flag not be assigned, and frames will be decoded/scaled by software, the buffer will be copied between HW encoder and user space.
• If the --record_path is assigned, the background recorder will start immediately after running the program. But the performance of Pi 3B is limited, if the --v4l2_format=mjpeg source resolution is above 640x368@15fps the HW codec will be unstable, stuck, or even crash. However, it can record smoothly at 960x480@30fps when the --v4l2_format=h264 and --enable_v4l2_dma flags are applied, even with two clients watching p2p streams simultaneously. Since the h264 source directly records into mp4 files without going through the decode/encode process.

Run as Linux Service

Set pi_webrtc to run as a daemon.

• Create /etc/systemd/system/webrtc.service, config sample:

  [Unit]
  Description= the webrtc service need signaling server first
  After=systemd-networkd.service farmer-api.service
  
  [Service]
  Type=simple
  WorkingDirectory=/home/pi/IoT/RaspberryPi_WebRTC/build
  ExecStart=/home/pi/IoT/RaspberryPi_WebRTC/build/pi_webrtc --fps=30 --width=1280 --height=720 --v4l2_format=h264 --enable_v4l2_dma --mqtt_username=hakunamatata --mqtt_password=wonderful --record_path=/home/pi/video/
  ExecStop=/bin/kill -s SIGTERM $MAINPID
  Restart=always
  RestartSec=20
    
  [Install]
  WantedBy=multi-user.target

• Enable and run the service

  sudo systemctl enable webrtc.service
  sudo systemctl start webrtc.service

Install the coturn (Optional)

If the cellular network is used, the coturn is required because the 5G NAT setting by ISP may block p2p. Or try some cloud service that provides TURN server.

1. Install

   sudo apt update
   sudo apt install coturn
   sudo systemctl stop coturn.service

2. Edit config sudo nano /etc/turnserver.conf, uncomment or modify below options

   listening-port=3478
   listening-ip=192.168.x.x
   relay-ip=192.168.x.x
   external-ip=174.127.x.x/192.168.x.x
   #verbose
   lt-cred-mech
   user=webrtc:webrtc
   realm=greenhouse
   no-tls
   no-dtls
   syslog
   no-cli

3. Set the port 3478 forwarding on the router/modem.
4. Start the service, sudo systemctl start coturn.service

Reference

• Version | WebRTC
• Building old revisions | WebRTC
• Using a custom clang binary | WebRTC
• Trickle ICE