Skip to content

NAF adapters comparison

Jump to bottom
Vincent Fretin edited this page Aug 8, 2021 · 1 revision

NAF adapters comparison

This document was written by Vincent Fretin @vincentfretin around June 2021 with his understanding at the time of writing. Although he tried hard to include useful information describing the different solutions, the document may contain mistakes or misunderstanding. Some solutions may have been updated in the meantime so some information may be out of date.

Currently maintained adapters are easyrtc and janus by the networked-aframe community.

Mesh, SFU, MCU

You can read the article WebRTC For The Curious that explains the difference between mesh, SFU and MCU in WebRTC.

The different topologies are also explained in this chapter of the WebRTC For The Curious book.

EasyRTC adapter

Easyrtc uses a mesh topology so you actually send your audio and video streams (voice and webcam or screen sharing) to each participant. Some users may have a very limited upload bandwidth like 0.6 MB/s, so this can be a real issue to upload the streams to 3 or 4 persons. In contrast, the janus adapter is a SFU, you upload only once your audio and video streams to a server and the server forwards the streams to all the participants.

Known limitations:

  • issue when you restart the node.js server, the participants doesn't reconnect properly #242
  • can only share the webcam, not any MediaStream with a video track #269
  • difficult to create an UI where you can select your mic from a list of available microphones because the open-easyrtc library asks for the mic for you. This is some difficulty to enable some options like echoCancellation: true, see open-easyrtc#47

Janus SFU adapter

Initially developed by mozilla for Hubs, now maintained by the networked-aframe community.

In a mesh topology, communications are end to end encrypted by nature with WebRTC because you connect directly to the other participant. With a SFU, specially with the current mozilla janus sfu, the communications are not end to end encrypted (e2ee). They are encrypted between a participant and the server, and between the server and another participant, but the server itself can see the streams in clear and record them. There is the new WebRTC Insertable Streams implemented now on Chrome that open the possibility to do e2ee with a SFU, see section below for different links on the subject.

It may be a feature or an issue depending on your needs.

Good things in this adapter:

  • auto reconnect the participants when the janus server restart
  • setLocalMediaStream api so you can share any audio and any video streams you want, call yourself getUserMedia for the mic or getDisplayMedia for screen sharing
  • private chat (when using websocket instead of datachannel for the reliable transport)
  • syncOccupants api combined to an external users source (Phoenix Presence) can allow for much greater number of users and creating private zones for example (need naf reliableTransport and unreliableTransport to not go through janus but like a Phoenix socket, and eventually a change to not create a publisher)
  • secure rooms with JWT to protect private rooms against eavesdropping (need a backend to create a room and generate the JWT associated with the room)

Current known limitations of the mozilla janus sfu:

  • it creates a RTCPeerConnection to the server for each participant. Some browsers may have limits to the number of RTCPeerConnection you can create, on Chrome you may start having issues with more than 30 connections https://bugs.chromium.org/p/chromium/issues/detail?id=825576 Creating a RTCPeerConnection to subscribe to each participant is a limitation of janus core, but this is fixed by the multistream branch https://github.com/meetecho/janus-gateway/pull/2211 What the multistream branch allows is to maintain only two RTCPeerConnection, one for sending your streams, and another to receive the streams of all participants. Once it is merged this will need lots of changes in the Rust code of janus sfu plugin and the javascript code in naf-janus-adapter. Other WebRTC stack like mediasoup doesn't have this limitation.
  • the RTCPeerConnection sdp offer/answer supports only one audio track and zero or one video track, so only one video stream can be used, you share your webcam or another video stream (may be a screen sharing or any MediaStream with a video track), not both.
  • the video codec used is hard coded to be the h264 codec, but you can change to use vp8 or vp9 if you want
  • no simulcast (upload several quality of your video stream and each participant subscribe to the quality they want), so be careful with the webcam or screen resolution you chose to send, all the subscribers may not be able to download the stream fast enough. Janus core supports simulcast (max 3 substreams), it's just the Rust plugin that doesn't implement it.

Hubs dialog adapter (mediasoup stack)

Hubs moved to the mediasoup stack (https://mediasoup.org and https://protoojs.org), this is also a SFU. Hubs is able now to have a lobby with 75 subscribers listening to the publishers' audio, and 24 publishers in the room. They recently added video simulcast, and the may allow to share both webcam and screen sharing in the future.

Each participant has only 2 RTCPeerConnections.

The backend part is a node.js app called dialog, the repo is here https://github.com/mozilla/dialog

You need to have a separate backend that create a JWT to be able to use that. And there may be some features missing to be used on a simple naf project. Hubs is using the Phoenix websocket (Reticulum) for the signaling part (who is connected in the room), chat and component updates, this is configured here https://github.com/mozilla/hubs/blob/d7b33f24f8a76b27462bb3f6f42051a55a913dc2/src/hub.js#L637-L638

The naf adapter is in the hubs repo here https://github.com/mozilla/hubs/blob/master/src/naf-dialog-adapter.js

One could extract this adapter to put it in an external repo (still under MPL licence) and adapt it to remove Hubs specific code so that it can work on a simple naf project.

The naf adapter currently calls some Hubs api (window.APP.store.update in enableMicrophone to update the React UI).

What's the difference between janus sfu plugin and janus videoroom plugin?

There are several plugins for janus. The janus videoroom plugin is a SFU for audio and video. Mozilla's janus-plugin-sfu is a SFU. The mozilla's janus adapter uses the janus-plugin-sfu and supports spatial audio. Janus also have an audiobrige plugin that acts as a MCU only for audio.

Mozilla's janus sfu plugin and janus videoroom plugin are two completely different plugins for janus.

janus videoroom plugin:

  • you can define the number of publishers (max_publishers) per room, and have some participants be watcher only (not publisher)
  • need to create the room and specify the video codec and profile for the room, and take care of destroying the room
  • no naf adapter

janus sfu plugin:

  • global config for all rooms (max_room_size equivalent to the max_publishers in videoroom)
  • auto creating and destroying the room (it's just an id really), h264 video codec hard coded
  • every participant is a publisher, you can start having some issue with more than 24 because of the number of sessions (WebRTC connections) the server needs to maintain, 24^2=576 sessions. You could have some listeners only to avoid creating sessions to participants that don't talk, this is somehow half supported, this could be used right now with an external users source like Phoenix Presence with an Elixir backend, see https://github.com/networked-aframe/naf-janus-adapter/pull/9 and https://github.com/mozilla/janus-plugin-sfu/issues/83 but with just a janus deployment, this is not possible, see https://github.com/mozilla/janus-plugin-sfu/issues/76

Are they any adapter for MCU topology?

No. MCU would mean there is no spatial audio in the room. You can't have positional audio with a MCU because the audio streams are mixed on the server and you get only one audio stream. But janus audiobridge plugin (MCU for audio) has a pending feature to allow some spatial audio, see https://github.com/meetecho/janus-gateway/pull/2446

Other adapters

Some people created naf adapters for different services. An example with croquet.io for the signaling part and simple-peer for the data and audio exchange. See https://twitter.com/ConcreteSciFi/status/1337433213917007880 and https://aframevr.slack.com/archives/C65MC3NSF/p1607704541105700 repository: https://github.com/zakaton/networked-aframe This adapter is in the category of mesh topology.

There were comments about others working on a jitsi adapter, but not publicly.

One could create a SFU in Go based on the Pion stack (see ion-sfu) and the associated adapter. This could be very similar to what janus-plugin-sfu in Rust is doing. Interesting interview of Pion main developer

End to end encryption with WebRTC Insertable Streams in various SFU

Clone this wiki locally