Rtmp features

[pedroSG94]

This discussion is for join al features or ideas related to RTMP. I will update the main comment with accepted features in progress

• Stream mode that support change video and audio source on fly: https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/tree/custombase

[maddygoround]

Smooth switching between RTMPDisplay and RTMPFile without disconnecting the rtmp connection.

[pedroSG94]

We need specify multiple things to allow it. For example:

If you use a file or display with different audio/video configuration (resolution, audio channels, samplerate, etc).
What should we do in each this case?

[maddygoround]

I haven't thought it through, but the idea is to treat each class camera1, camera2 ,display and file as a scenes, like scenes in OBS, and then you are allowed to switch between the scenes.

[pedroSG94]

I see.
This seems possible but it is a large feature.
In this case video source should be re scaled to fit stream resolution and audio source could be forced to configure with same channels and samplerate or it will fail.
I'm going to think a bit more about it.

[maddygoround]

We are trying out the suggestion you made in following two issues

#624
#548

I hope it get us somewhere.

[maddygoround]

@pedroSG94 did you put any thoughts on it? Will you be willing to help us build such feature?

[pedroSG94]

I'm on it but It will take time because I have no time this days to develop

[maddygoround]

I understand. Thanks for quick update.

[pedroSG94]

I will create a branch soon where you can check the actual state

[maddygoround]

Thanks. By the way I have sent you a mail on your personal email.

[pedroSG94]

branch for this feature:
https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/tree/custombase

[maddygoround]

Thanks for quick work @pedroSG94.

[maddygoround]

@pedroSG94 We started utilizing the updated code. I will keep you posted on how it's coming out.

[pedroSG94]

If you mean custombase branch. It is under develop. You can test it but it will fail for sure

[maddygoround]

Yes I understand that. Our point was to try out and see how far we can go. btw it didn't failed entirely but it do stop the streaming as soon as the app goes in background, which is understandable knowing there is lot to be implemented.

[pedroSG94]

Yes, it stop when you go to background but I did it to keep example easy to test the new features but is should work if you change the code a bit.
If you remove this line the stream should continue in background with app closed:
https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/custombase/app/src/main/java/com/pedro/rtpstreamer/custom/CustomModeRtmp.kt#L76
If you want test change video source change this lines:
https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/custombase/app/src/main/java/com/pedro/rtpstreamer/custom/CustomModeRtmp.kt#L56
https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/custombase/app/src/main/java/com/pedro/rtpstreamer/custom/CustomModeRtmp.kt#L64
To changeVideoSource method and you can test it.

Actually I need develop this:

• Implement from file source (video and audio)
• Fix scale and rotation in video source.
• Fix record video.
• Fix a minor bug detected with audio and video source change on fly.
• Fix changeGlInterface bugs.

After that I only need test it well and debug.

[maddygoround]

Looking forward to all the development. We are currently trying to switch between filesource and display without actually showing the file source to the user.

so on the stream you can see the switch from file to display and viceversa but as a user of a device you only see the screen-display. We are trying to do this via taking 1x1 pixel surface view on display source, so it's barley visible to user.

The current challenge is with the audio. We somehow have to separate the audio source of display from the audio source of the video running inside the 1x1 surface view.

This is just to put forward my usecase.

[maddygoround]

Hi Pedro,

We are on a work around with display stream (with video)

to do this we start displaying the video on overlay with display stream and tries to get audio pcmbuffer from file stream's audio decoder, then replace the pcm buffer of display stream with file streams PCMBuffer in "setCustomAudioEffect" (Not starting the FileStream but only Audio Decoder).

Size of File stream's pcmBuffer is 4096, we are converting that into 5127 as that's the size of Display stream's pcmBuffer.
Size of Display stream's pcmBuffer is 5127 in setCustomAudioEffect --> process

When tries to replace the pcmBuffer in setCustomAudioEffect --> process, process get stopped after 4 iterations, and then no audio get transmitted.