Any know how to deal with the H.264 video in a better way？

[hotDOGEEE]

Could we have any way to make a simple DEMO by using the h264 player from this project?
I've been building a platform. Right now I'm trying to use the WS-SCRCPY service.
Dealing with the video stream is a hard thing for me right now.
Will be apppreciate if you can show me some samples or teach me some details about the frame.

[drauggres]

Hi.
All players in this project are based on other open source projects (except WebCodecs).
If you are only interested in h264-video, you should use them directly, there is nothing special in ws-scrcpy.
https://github.com/NetrisTV/ws-scrcpy#screen-casting

One more thing: all this players work with h264 NALUs, i.e raw h264-stream without container like mp4/mpeg-ts/flv.

[hotDOGEEE]

Hi,
Thank you for your reply. Actually I would like to use the h264 player directly. As a beginner in web development. I’ve tried to replace a web player, still not that good to me.
Now the ideal sample be like this:

<video id="videowindow" width="1280" height="720">
<script type="text/javascript" src="/wsh264player.js">
	window.onload = function() {
		var videowindow = document.getElmentById("videowindow"),
		whp = new wsh264player();
                whp.setaddress("ws://localhost:8000/?action=proxy-adb&remote=tcp%3A8886&udid=MFH6JBV4OFGY65J7")
		whp.attachMedia(videowindow, "ch1");
	};
</script>

The idea came from this project:
https://github.com/ChihChengYang/wfs.js
Which contains one player matching a front-end interface.

I prefer the interface from WS-SCRCPY which can handle the devices and switch the screens.
In a word, I would like to use the web player from WS-SCRCPY and this player is able to call the parameters as the interface.

[sbfkcel]

ws-scrcpy seems to already handle the encoding on the server side? So I didn't look closely. In the end I chose to use a modified version of ws-scrcpy directly.

I tried to decode the raw data stream emitted by scrcpy and it didn't work very well.
The general idea is to extract all NALUs of the h264 data stream (separated by 0,0,0,1 or 0,0,1), and splicing each NALU into frame data according to the encoding rules (refer to ws-scrcpy). Use the decoder (WebCodecs) to decode the frame data and render into the canvas.

Maybe I'm not very familiar with H264 encoding, the intercepted data can only render a part or the first few frames, and there will be errors.

I hope the author can share some processing methods about H264. Thank you so much.

@hotDOGEEE wfs.js can only render a part of the frame, and it is bloated.

[hotDOGEEE]

ws-scrcpy 似乎已经处理了服务器端的编码？所以我没有仔细看。最后我选择直接使用修改版的ws-scrcpy。

我试图解码 scrcpy 发出的原始数据流，但效果不佳。 大致思路是提取h264数据流的所有NALU（用0,0,0,1or分隔0,0,1），按照编码规则（参考ws-scrcpy）将每个NALU拼接成帧数据。使用解码器 (WebCodecs) 解码帧数据并渲染到画布中。

可能我对H264编码不是很熟悉，截取的数据只能渲染一部分或者前几帧，会出现错误。

希望作者能分享一些关于H264的处理方法。太感谢了。

@hotDOGEEE wfs.js 只能渲染部分帧，而且臃肿。

嗯，或者关于ws的图片和操作，具体的通信协议和关键字解析能出一个类似接口文档的内容，会很方便

[sbfkcel]

我在与scrcpy服务端连接层做了与NetrisTV修改版本相似的工作。对scrcpy数据进行了再加工，这样的好处是不需要对原始的scrcpy进行改动。之后我可能会出一个原始的scrcpy数据流规则文章介绍或开源出来。

[drauggres]

之后我可能会出一个原始的scrcpy数据流规则文章介绍或开源出来。

https://github.com/yume-chan/ya-webadb/tree/master/libraries/scrcpy

[sbfkcel]

之后我可能会出一个原始的scrcpy数据流规则文章介绍或开源出来。

https://github.com/yume-chan/ya-webadb/tree/master/libraries/scrcpy

我参考了ya-webadb