Move the player window out of the qvh process

[danielpaulus]

Hey all, I am throwing ideas on how to achieve this. Having the gstreamer player run inside qvh is a major blocker right now to make progress on the project.

My first attempt to get a good player working was writing a golang/cgo wrapper for gstreamer (OSS multimediaframework), that works okayish and people have tons of issues with it.
I decided I don’t want the actual video player window to run inside my process now, because it is not super stable (works locally but, people having tons of issues with it).
So I was thinking UDP, but UDP is actually not fast enough. The challenge is getting huge, high quality raw audio and h264 with zero latency from one process to the other. I hacked on UDP until 1am and then gave up with a choppy ffmpeg player as a result :grumpycat:

Next up, I want to try IPC through /dev/shm/

[drauggres]

Hi. I found this project when I was looking for scrcpy analogue for iOS.
Basically I needed to control remotely dozen smartphones. I had modified versions of scrcpy and qvh with integrated WebSocket servers and "unified" web client. There were no problems with transferring video stream (no audio) from qvh to a browser over WebSocket (latency was near 60ms, which is not zero, but totally usable).

As for separating player from the main app, I totally agree. Gstreamer was good as PoC, but not really usefull and it brought many complications.
IMHO FFmpeg is a way to go. Maybe you can even adapt qvh to work with scrcpy player or write something similar.

[danielpaulus]

I agree with your suggestions. Do you have some good insights or ideas how I can get the data over to ffmpeg? I don't have a lot of experience with it and yesterday tried to send the data using udp to ffmpeg and then render a player window with ffplay. That was too slow and did not really work well.

[danielpaulus]

(PS this is also the biggest blocker in finally getting the pause/resume feature done)

[drauggres]

No, sorry. As I mentioned I used web browser as a client and almost never touched original scrcpy client/player (or ffmpeg for that matter).
But scrcpy (which uses ffmpeg) is under permissive license and it is doing pretty much the same job.
https://github.com/Genymobile/scrcpy/blob/master/DEVELOP.md#client

[nnnpa31]

I agree with your suggestions. Do you have some good insights or ideas how I can get the data over to ffmpeg? I don't have a lot of experience with it and yesterday tried to send the data using udp to ffmpeg and then render a player window with ffplay. That was too slow and did not really work well.

@danielpaulus

I refer to this method and use go-astiav to achieve the decoding of h264 bare streams, I haven't actually played it yet, but I don't think the delay should be more than two frames.