Incorrect DTLS handshake when acting as a server

[xou816]

I am writing a WebRTC peer using this crate, making an offer and acting as a server during the TLS handshake. This peer is meant to talk with an instance of Kurento Media Server.

The handshake fails with the following error:

[2024-03-12T13:56:36Z TRACE webrtc_dtls::conn] server: <- Alert LevelFatal: IllegalParameter
[2024-03-12T13:56:36Z TRACE webrtc_dtls::conn] server: read_and_buffer return err: Alert is Fatal or Close Notify

Upon further inspection, it looks like the server (using webrtc-rs, ip 172.18.0.4) sends two Hello Verify Requests, and also sends Flight 4 right after the second Hello Verify Request:

The other peer (172.18.0.3) responds correctly to the first Hello Verify Request.

It seems that we end up in an inconsistent state, where Flight 2 is erroneously sent a second time before waiting for a response from the client, but Flight 4 is sent as well once the Client Hello is acknowledged. Some relevant logs:

[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 0: Preparing
[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 0: Sending
[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 0: Waiting
[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 0 result alert:None, err:None
[2024-03-12T13:43:12Z TRACE webrtc_dtls::conn] Recv [handshake:server] -> ClientHello (epoch: 0, seq: 0)
[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 0 -> Flight 2
[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 2: Preparing
[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 2: Sending
[2024-03-12T13:43:12Z TRACE webrtc_dtls::conn] Send [handshake:server] -> HelloVerifyRequest (epoch: 0, seq: 0)
[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 2: Waiting
[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 2 -> Flight 2
[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 2: Preparing
[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 2: Sending
[2024-03-12T13:43:12Z TRACE webrtc_dtls::conn] Send [handshake:server] -> HelloVerifyRequest (epoch: 0, seq: 0)
[2024-03-12T13:43:12Z TRACE webrtc_dtls::conn] Recv [handshake:server] -> ClientHello (epoch: 0, seq: 1)
[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 2: Waiting
[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 2 -> Flight 4
[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 4: Preparing
[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 4: Sending
[2024-03-12T13:43:12Z TRACE webrtc_dtls::conn] Send [handshake:server] -> ServerHello (epoch: 0, seq: 1)
[2024-03-12T13:43:12Z TRACE webrtc_dtls::conn] Send [handshake:server] -> Certificate (epoch: 0, seq: 2)
[2024-03-12T13:43:12Z TRACE webrtc_dtls::conn] Send [handshake:server] -> ServerKeyExchange (epoch: 0, seq: 3)
[2024-03-12T13:43:12Z TRACE webrtc_dtls::conn] Send [handshake:server] -> CertificateRequest (epoch: 0, seq: 4)
[2024-03-12T13:43:12Z TRACE webrtc_dtls::conn] Send [handshake:server] -> ServerHelloDone (epoch: 0, seq: 5)
[2024-03-12T13:43:12Z TRACE webrtc_dtls::handshaker] [handshake:server] Flight 4: Waiting
[2024-03-12T13:43:12Z TRACE webrtc_dtls::conn] server: <- Alert LevelFatal: IllegalParameter
[2024-03-12T13:43:12Z TRACE webrtc_dtls::conn] server: read_and_buffer return err: Alert is Fatal or Close Notify

I am not sure what is specific about my setup, I am working from the examples in the repo to assert if this crate fits my needs. I could try to provide a minimal setup but it involves a Kurento deployment and a custom signaling server.

I've tried using engine.set_answering_dtls_role(DTLSRole::Client) as a workaround, to no avail.

[xou816]

I've bypassed Flight 2 locally for a quick test, I still encounter the IllegalParameter alert, so the issue might be something else. I'll leave the issue open in case the behavior described above is incorrect despite that.