Conference room data channel only reports audio updates

[VanIseghemThomas]

Title says it all. I'm currently implementing conference streaming for the Unity platform and everything works like a charm except one aspect. When opening a data channel I am only able to retrieve UPDATE_AUDIO_LEVEL and AUDIO_TRACK_ASSIGNMENT event messages. Causing me to have problems mapping the video tracks of joined users since the I cant get ahold of the VIDEO_TRACK_ASSIGNMENT_LIST for example, even when explicitly querying it using the signaling. This is the case for all other features such as camera, mic mute, chat events etc.

My first thought was that I did something wrong opening the data channel, but I'm clearly getting the audio events so there is in fact a data channel. Any obvious things that could cause this?

[mustafaboleken]

Hi @VanIseghemThomas

You can use the TRACK_LIST_UPDATED message to be able to get the updates. VIDEO_TRACK_ASSIGNMENT_LIST triggerers only your video track assignment change but it didn't trigger everytime someone enters in a room.

For example, if the maximum video track count is 3, there is 5 participants in the room and a new participant is joined, you won't get VIDEO_TRACK_ASSIGNMENT_LIST.

You can stick with TRACK_LIST_UPDATED and broadcast_object callbacks. For more information: https://github.com/ant-media/conference-call-application/blob/main/react/src/pages/AntMedia.js#L1379 and https://github.com/ant-media/conference-call-application/blob/main/react/src/pages/AntMedia.js#L699

[VanIseghemThomas]

Hi @mustafaboleken

Thanks for checking up!
IIRC one of the things I observed from logs is that the ICE connection state starts at Connected, returns all the audio updates over the data channel, but then after a little while (let's say about 15 sec.) it goes into the state Completed but does not return any data anymore. Maybe that can be a clue for something?

From the WebRTC docs for the web which I assume Unity ported it's implementation from, it states:

connected
A usable pairing of local and remote candidates has been found for all components of the connection, and the connection has been established. It is possible that gathering is still underway, and it is also possible that the ICE agent is still checking candidates against one another looking for a better connection to use.

completed
The ICE agent has finished gathering candidates, has checked all pairs against one another, and has found a connection for all components.

A theory I have is that it switches to another connection causing the data channel to close, but I'm not really an expert at WebRTC.

[mustafaboleken]

Yes, we are using the same implementation like the documentation that you sent. ICE connection state must be in the Connected state. It looks like this is the reason that you cannot get data channel messages. Can you send me your ant media server logs and sdp information?

[burak-58]

We may make a call to debug it together.

[VanIseghemThomas]

@mustafaboleken @burak-58 we are currently really pushing towards a release and currently our trail license expired. We are looking to get a full license though. The next few weeks will be hard to continue on this, the release is planned for the start of June. After those I will get back to you guys and we can plan a moment to sit together and debug this. Already thanks in advance!

[mustafaboleken]

Thanks for letting us know. Good luck with the release.