RTC between 2 .NET Clients, SignalR as WebSocket layer

[brandonh-msft]

Hey folks,

I have the following architecture in a .NET 8.0 environment:

• SignalR Hub (ASP.NET Core)
• Worker console app (text in, text out)
• Orchestrator app (headless)

What I want to do is for the End User and the Orchestrator to establish an RTC data connection

I start by sending a message through SignalR to the Orchestrator, from the End User:

var directConnInfo = new RTCPeerConnection();
var chan = await directConnInfo.createDataChannel("streamingResponse");
var directConnOffer = directConnInfo.createOffer();

await signalr.SendAsync(Constants.SignalR.Functions.StartDirectConnection, Constants.SignalR.Users.Orchestrator, directConnOffer);

then, in SignalR, I forward this on to the Orchestrator

[HubMethodName(Constants.SignalR.Functions.StartDirectConnection)]
public async Task StartDirectConnectionAsync(string toUser, RTCSessionDescriptionInit session)
{
    // code that gets connectionId for `toUser`
    await this.Clients.Client(userConn).SendAsync(Constants.SignalR.Functions.StartDirectConnection, session, default);
}

In the Orchestrator, then, I field this request:

this.SignalR.On<string>(Constants.SignalR.Functions.StartDirectConnection, async d =>
{
    _log.LogTrace("Received request to open RTC connection {rtcInit}", d);

    var result = _peerConnection.setRemoteDescription(d);
    _log.LogTrace("setRemoteDescription {result}", result);
});

which returns SetDescriptionResultEnum.OK as expected.

What I'm not sure about is what I do next. I know I need to addIce stuff... but I'm not sure how to create the IceDescriptionInit value that gets sent along the wire as it shows in Step 5 of these docs as when onicecandidate is fired, my signalingState is never RTCSignalingState.have_remote_offer

I've subscribed to the oniceconnectionstatechange and onconnectionstatechange values and in both the End User and the Router and this is what I end up seeing:

End User:

info: Worker[0] Connecting to server...
dbug: Worker[0] ICE Candidate (closed): {"candidate":"candidate:3088468610 1 udp 2113937663 192.168.1.16 60384 typ host generation 0","sdpMid":"0","sdpMLineIndex":0,"usernameFragment":"CKLT"}
dbug: Worker[0] ICE Candidate (closed): {"candidate":"candidate:1706853177 1 udp 2113940223 fd02:8850:6de9:6778:254d:c99d:b736:3596 60384 typ host generation 0","sdpMid":"0","sdpMLineIndex":0,"usernameFragment":"CKLT"}
dbug: Worker[0] ICE Candidate (closed): {"candidate":"candidate:1561383423 1 udp 2113940223 fd02:8850:6de9:6778:7e03:72ce:b9e3:b21e 60384 typ host generation 0","sdpMid":"0","sdpMLineIndex":0,"usernameFragment":"CKLT"}

Orchestrator:

dbug: Orchestrator[0] => ConnectToSignalRAsync Connecting to SignalR hub...
dbug: Orchestrator[0] Orchestrator - ICE Candidate (closed): {"candidate":"candidate:1561383423 1 udp 2113940223 fd02:8850:6de9:6778:7e03:72ce:b9e3:b21e 58296 typ host generation 0","sdpMid":"0","sdpMLineIndex":0,"usernameFragment":"TPOC"}
dbug: Orchestrator[0] Orchestrator - ICE Candidate (closed): {"candidate":"candidate:1706853177 1 udp 2113940223 fd02:8850:6de9:6778:254d:c99d:b736:3596 58296 typ host generation 0","sdpMid":"0","sdpMLineIndex":0,"usernameFragment":"TPOC"}
dbug: Orchestrator[0] Orchestrator - ICE Candidate (closed): {"candidate":"candidate:3088468610 1 udp 2113937663 192.168.1.16 58296 typ host generation 0","sdpMid":"0","sdpMLineIndex":0,"usernameFragment":"TPOC"}
info: Orchestrator[0] Awaiting question...
trce: Orchestrator[0] => ConnectToSignalRAsync Received request to open RTC connection {"type":"offer","sdp":"v=0\r\no=- 55652 0 IN IP4 127.0.0.1\r\ns=sipsorcery\r\nt=0 0\r\na=group:BUNDLE 0\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:CKLT\r\na=ice-pwd:KRPWOVAXIMMNCHTOFZDBSVEL\r\na=fingerprint:sha-256 EA:E8:9D:B2:0B:60:77:20:B1:E8:23:17:AB:79:F6:7F:F1:A4:6E:9D:C5:37:64:DF:E4:FF:16:47:25:51:3D:80\r\na=setup:actpass\r\na=candidate:3088468610 1 udp 2113937663 192.168.1.16 60384 typ host generation 0\r\na=candidate:1706853177 1 udp 2113940223 fd02:8850:6de9:6778:254d:c99d:b736:3596 60384 typ host generation 0\r\na=candidate:1561383423 1 udp 2113940223 fd02:8850:6de9:6778:7e03:72ce:b9e3:b21e 60384 typ host generation 0\r\na=ice-options:ice2,trickle\r\na=mid:0\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=end-of-candidates\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n"}
dbug: Orchestrator[0] => ConnectToSignalRAsync ICE connection state: checking
trce: Orchestrator[0] => ConnectToSignalRAsync setRemoteDescription OK
dbug: Orchestrator[0] ICE connection state: failed
dbug: Orchestrator[0] Connection state: failed

[sipsorcery]

Hi Brandon,

So the Orchestrator has its own RTCPeerConnection instance somewhere? I'm guessing it's the one in this line of code.

var result = _peerConnection.setRemoteDescription(d);

Assuming that's the case are you performing step 4 and sending an SDP answer from the Orchestrator back to the End User? The SDP protocol is an offer/answer where the one party generates the offer, sends it to the other party who generates an answer and sends it back. Until both ends have the SDP from the remote party the ICE connection attempts cannot proceed.

Once the SDP exhcnage is complete then step 5 referes to echancing ICE candidates. This is a one way exchange int hat each party sends its candidates to the remote party but does not need an answer. As the ICE candidates are generated you will need to send them via SignalR to the remote party. The candidates can sometimes take a second or two to generate as peers try and determine what their public IP address is . negotiate with a TURN server etc.

Hope that helps.
Aaron