Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug: Local track wont show up in the answer #563

Open
magniff opened this issue May 4, 2024 · 0 comments
Open

Bug: Local track wont show up in the answer #563

magniff opened this issue May 4, 2024 · 0 comments

Comments

@magniff
Copy link

magniff commented May 4, 2024

Version: 0.11.0
Looks like there's a regression in 0.11.0 compared to 0.8.0, that i am currently using, here's a minimal code that reproduces the potential bug:

use webrtc::{
    api::{
        interceptor_registry::register_default_interceptors,
        media_engine::{MediaEngine, MIME_TYPE_VP8},
        APIBuilder,
    },
    ice_transport::ice_server::RTCIceServer,
    interceptor::registry::Registry,
    peer_connection::{
        configuration::RTCConfiguration, sdp::session_description::RTCSessionDescription,
        RTCPeerConnection,
    },
    rtp_transceiver::rtp_codec::RTCRtpCodecCapability,
    track::track_local::{track_local_static_rtp::TrackLocalStaticRTP, TrackLocal},
};

static OFFER: &str = stringify!(
    {
        "type": "offer",
        "sdp": "v=0\r\no=- 4087711290343265416 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS 72a14f63-48f0-48f0-ba8d-ab1ceed4ce87\r\nm=video 37567 UDP/TLS/RTP/SAVPF 96 97 102 103 104 105 106 107 108 109 127 125 39 40 45 46 98 99 100 101 112 113 114\r\nc=IN IP4 172.19.0.1\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=candidate:4172267714 1 udp 2122194687 172.19.0.1 37567 typ host generation 0 network-id 1\r\na=candidate:902995773 1 udp 2122129151 100.67.97.111 46721 typ host generation 0 network-id 2\r\na=candidate:4004585221 1 udp 2122063615 10.253.222.31 49608 typ host generation 0 network-id 4 network-cost 10\r\na=candidate:2521055259 1 udp 2122265343 fd7a:115c:a1e0::9103:616f 47292 typ host generation 0 network-id 3\r\na=candidate:100996182 1 tcp 1518214911 172.19.0.1 9 typ host tcptype active generation 0 network-id 1\r\na=candidate:3413658537 1 tcp 1518149375 100.67.97.111 9 typ host tcptype active generation 0 network-id 2\r\na=candidate:270257041 1 tcp 1518083839 10.253.222.31 9 typ host tcptype active generation 0 network-id 4 network-cost 10\r\na=candidate:1760466063 1 tcp 1518285567 fd7a:115c:a1e0::9103:616f 9 typ host tcptype active generation 0 network-id 3\r\na=ice-ufrag:Na8q\r\na=ice-pwd:BSiW7p34fAVU7jZnpfW4CP0m\r\na=ice-options:trickle\r\na=fingerprint:sha-256 A0:1D:AF:C0:08:4D:F3:AB:FD:64:4E:FD:CF:66:2E:E2:3B:C2:30:82:26:0A:84:AA:5A:FD:06:B7:F5:93:B1:75\r\na=setup:actpass\r\na=mid:0\r\na=extmap:1 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:3 urn:3gpp:video-orientation\r\na=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\na=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\na=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space\r\na=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=sendrecv\r\na=msid:72a14f63-48f0-48f0-ba8d-ab1ceed4ce87 5c627cc7-c0be-4f2d-8760-8515343a5c06\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 VP8/90000\r\na=rtcp-fb:96 goog-remb\r\na=rtcp-fb:96 transport-cc\r\na=rtcp-fb:96 ccm fir\r\na=rtcp-fb:96 nack\r\na=rtcp-fb:96 nack pli\r\na=rtpmap:97 rtx/90000\r\na=fmtp:97 apt=96\r\na=rtpmap:102 H264/90000\r\na=rtcp-fb:102 goog-remb\r\na=rtcp-fb:102 transport-cc\r\na=rtcp-fb:102 ccm fir\r\na=rtcp-fb:102 nack\r\na=rtcp-fb:102 nack pli\r\na=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f\r\na=rtpmap:103 rtx/90000\r\na=fmtp:103 apt=102\r\na=rtpmap:104 H264/90000\r\na=rtcp-fb:104 goog-remb\r\na=rtcp-fb:104 transport-cc\r\na=rtcp-fb:104 ccm fir\r\na=rtcp-fb:104 nack\r\na=rtcp-fb:104 nack pli\r\na=fmtp:104 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f\r\na=rtpmap:105 rtx/90000\r\na=fmtp:105 apt=104\r\na=rtpmap:106 H264/90000\r\na=rtcp-fb:106 goog-remb\r\na=rtcp-fb:106 transport-cc\r\na=rtcp-fb:106 ccm fir\r\na=rtcp-fb:106 nack\r\na=rtcp-fb:106 nack pli\r\na=fmtp:106 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:107 rtx/90000\r\na=fmtp:107 apt=106\r\na=rtpmap:108 H264/90000\r\na=rtcp-fb:108 goog-remb\r\na=rtcp-fb:108 transport-cc\r\na=rtcp-fb:108 ccm fir\r\na=rtcp-fb:108 nack\r\na=rtcp-fb:108 nack pli\r\na=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f\r\na=rtpmap:109 rtx/90000\r\na=fmtp:109 apt=108\r\na=rtpmap:127 H264/90000\r\na=rtcp-fb:127 goog-remb\r\na=rtcp-fb:127 transport-cc\r\na=rtcp-fb:127 ccm fir\r\na=rtcp-fb:127 nack\r\na=rtcp-fb:127 nack pli\r\na=fmtp:127 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d001f\r\na=rtpmap:125 rtx/90000\r\na=fmtp:125 apt=127\r\na=rtpmap:39 H264/90000\r\na=rtcp-fb:39 goog-remb\r\na=rtcp-fb:39 transport-cc\r\na=rtcp-fb:39 ccm fir\r\na=rtcp-fb:39 nack\r\na=rtcp-fb:39 nack pli\r\na=fmtp:39 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=4d001f\r\na=rtpmap:40 rtx/90000\r\na=fmtp:40 apt=39\r\na=rtpmap:45 AV1/90000\r\na=rtcp-fb:45 goog-remb\r\na=rtcp-fb:45 transport-cc\r\na=rtcp-fb:45 ccm fir\r\na=rtcp-fb:45 nack\r\na=rtcp-fb:45 nack pli\r\na=fmtp:45 level-idx=5;profile=0;tier=0\r\na=rtpmap:46 rtx/90000\r\na=fmtp:46 apt=45\r\na=rtpmap:98 VP9/90000\r\na=rtcp-fb:98 goog-remb\r\na=rtcp-fb:98 transport-cc\r\na=rtcp-fb:98 ccm fir\r\na=rtcp-fb:98 nack\r\na=rtcp-fb:98 nack pli\r\na=fmtp:98 profile-id=0\r\na=rtpmap:99 rtx/90000\r\na=fmtp:99 apt=98\r\na=rtpmap:100 VP9/90000\r\na=rtcp-fb:100 goog-remb\r\na=rtcp-fb:100 transport-cc\r\na=rtcp-fb:100 ccm fir\r\na=rtcp-fb:100 nack\r\na=rtcp-fb:100 nack pli\r\na=fmtp:100 profile-id=2\r\na=rtpmap:101 rtx/90000\r\na=fmtp:101 apt=100\r\na=rtpmap:112 red/90000\r\na=rtpmap:113 rtx/90000\r\na=fmtp:113 apt=112\r\na=rtpmap:114 ulpfec/90000\r\na=ssrc-group:FID 2651445778 2834855134\r\na=ssrc:2651445778 cname:TEJyWJIUChiRSXVY\r\na=ssrc:2651445778 msid:72a14f63-48f0-48f0-ba8d-ab1ceed4ce87 5c627cc7-c0be-4f2d-8760-8515343a5c06\r\na=ssrc:2834855134 cname:TEJyWJIUChiRSXVY\r\na=ssrc:2834855134 msid:72a14f63-48f0-48f0-ba8d-ab1ceed4ce87 5c627cc7-c0be-4f2d-8760-8515343a5c06\r\n"
    }
);

pub async fn generate_sdp_answer(
    peer_connection: &RTCPeerConnection,
) -> Result<RTCSessionDescription, webrtc::Error> {
    // Create an answer
    let answer = peer_connection.create_answer(None).await?;
    let mut gather_complete = peer_connection.gathering_complete_promise().await;
    peer_connection
        .set_local_description(answer.clone())
        .await?;
    let _ = gather_complete.recv().await;
    Ok(answer)
}

#[tokio::main]
async fn main() {
    let mut media_engine = MediaEngine::default();
    media_engine.register_default_codecs().unwrap();

    let config = RTCConfiguration {
        ice_servers: vec![RTCIceServer {
            urls: vec!["stun:stun.l.google.com:19302".to_owned()],
            ..Default::default()
        }],
        ..Default::default()
    };
    let peer_connection = APIBuilder::new()
        .with_interceptor_registry(
            register_default_interceptors(Registry::new(), &mut media_engine).unwrap(),
        )
        .with_media_engine(media_engine)
        .build()
        .new_peer_connection(config)
        .await
        .unwrap();

    let remote_desc = serde_json::from_str::<RTCSessionDescription>(OFFER).unwrap();
    let output_track = std::sync::Arc::new(TrackLocalStaticRTP::new(
        RTCRtpCodecCapability {
            mime_type: MIME_TYPE_VP8.to_owned(),
            ..Default::default()
        },
        "my-id".into(),
        "my-stream-id".into(),
    ));

    peer_connection
        .set_remote_description(remote_desc)
        .await
        .unwrap();

    let _sender = peer_connection
        .add_track(
            std::sync::Arc::clone(&output_track) as std::sync::Arc<dyn TrackLocal + Send + Sync>
        )
        .await
        .unwrap();

    let answer = generate_sdp_answer(&peer_connection).await.unwrap();
    println!("{}", answer.sdp);
}

Description: I've created a thread local track, added it to the peer connection and tried to get it showing up within my webrtc answer. I am using a simple OFFER, that i grabbed from by browser - it offers a VP8 video track from the built in web cam.

Expected answer (behavior of 0.8.0):

v=0
o=- 6361093872056348700 55404655 IN IP4 0.0.0.0
s=-
t=0 0
a=fingerprint:sha-256 3E:1F:7E:10:1B:BC:A4:62:42:59:DA:20:2B:55:E6:79:81:F4:EA:E5:D0:12:A0:73:CF:E7:10:B3:FE:0B:36:1A
a=group:BUNDLE 0
m=video 9 UDP/TLS/RTP/SAVPF 96 102 104 106 108 98 114
c=IN IP4 0.0.0.0
a=setup:active
a=mid:0
a=ice-ufrag:BFnQIsegXCmQpCln
a=ice-pwd:buqQNduGQTQgTdWzKuyjiXpwfHHRQaqJ
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb 
a=rtcp-fb:96 transport-cc 
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack 
a=rtcp-fb:96 nack pli
a=rtpmap:102 H264/90000
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtcp-fb:102 goog-remb 
a=rtcp-fb:102 transport-cc 
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack 
a=rtcp-fb:102 nack pli
a=rtpmap:104 H264/90000
a=fmtp:104 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtcp-fb:104 goog-remb 
a=rtcp-fb:104 transport-cc 
a=rtcp-fb:104 ccm fir
a=rtcp-fb:104 nack 
a=rtcp-fb:104 nack pli
a=rtpmap:106 H264/90000
a=fmtp:106 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtcp-fb:106 goog-remb 
a=rtcp-fb:106 transport-cc 
a=rtcp-fb:106 ccm fir
a=rtcp-fb:106 nack 
a=rtcp-fb:106 nack pli
a=rtpmap:108 H264/90000
a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
a=rtcp-fb:108 goog-remb 
a=rtcp-fb:108 transport-cc 
a=rtcp-fb:108 ccm fir
a=rtcp-fb:108 nack 
a=rtcp-fb:108 nack pli
a=rtpmap:98 VP9/90000
a=fmtp:98 profile-id=0
a=rtcp-fb:98 goog-remb 
a=rtcp-fb:98 transport-cc 
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack 
a=rtcp-fb:98 nack pli
a=rtpmap:114 ulpfec/90000
a=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=ssrc:741514651 cname:my-stream-id
a=ssrc:741514651 msid:my-stream-id my-id
a=ssrc:741514651 mslabel:my-stream-id
a=ssrc:741514651 label:my-id
a=sendrecv

Actual (what i am getting in 0.11.0):

v=0
o=- 6867872315504930623 664034582 IN IP4 0.0.0.0
s=-
t=0 0
a=fingerprint:sha-256 4A:93:79:52:B6:A4:86:A4:BE:A6:D6:80:B9:DF:53:0D:12:B9:02:FC:52:CB:53:CE:9A:AC:77:60:5A:38:F0:8A
a=group:BUNDLE 0
m=video 9 UDP/TLS/RTP/SAVPF 96 102 104 106 108 45 98 114
c=IN IP4 0.0.0.0
a=setup:active
a=mid:0
a=ice-ufrag:JoSKpsaSlxaZkAnd
a=ice-pwd:ICVdBZBnZAwqRopxdBEhTMbpkZhAsHGa
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb 
a=rtcp-fb:96 transport-cc 
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack 
a=rtcp-fb:96 nack pli
a=rtpmap:102 H264/90000
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtcp-fb:102 goog-remb 
a=rtcp-fb:102 transport-cc 
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack 
a=rtcp-fb:102 nack pli
a=rtpmap:104 H264/90000
a=fmtp:104 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtcp-fb:104 goog-remb 
a=rtcp-fb:104 transport-cc 
a=rtcp-fb:104 ccm fir
a=rtcp-fb:104 nack 
a=rtcp-fb:104 nack pli
a=rtpmap:106 H264/90000
a=fmtp:106 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtcp-fb:106 goog-remb 
a=rtcp-fb:106 transport-cc 
a=rtcp-fb:106 ccm fir
a=rtcp-fb:106 nack 
a=rtcp-fb:106 nack pli
a=rtpmap:108 H264/90000
a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
a=rtcp-fb:108 goog-remb 
a=rtcp-fb:108 transport-cc 
a=rtcp-fb:108 ccm fir
a=rtcp-fb:108 nack 
a=rtcp-fb:108 nack pli
a=rtpmap:45 AV1/90000
a=fmtp:45 level-idx=5;profile=0;tier=0
a=rtcp-fb:45 goog-remb 
a=rtcp-fb:45 transport-cc 
a=rtcp-fb:45 ccm fir
a=rtcp-fb:45 nack 
a=rtcp-fb:45 nack pli
a=rtpmap:98 VP9/90000
a=fmtp:98 profile-id=0
a=rtcp-fb:98 goog-remb 
a=rtcp-fb:98 transport-cc 
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack 
a=rtcp-fb:98 nack pli
a=rtpmap:114 ulpfec/90000
a=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=recvonly

Explanation: 0.11.0's answer is missing this part

a=ssrc:741514651 cname:my-stream-id
a=ssrc:741514651 msid:my-stream-id my-id
a=ssrc:741514651 mslabel:my-stream-id
a=ssrc:741514651 label:my-id

In other words, the local track, that i've just created wont show up. Am i doing it wrong? Is there a workaround for that?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@magniff