Possible to share developer notes of how scrcpy works

[krishtoautomate]

scrcpy forwarding video is port in h264 format is understood. didn't get how touches are sent --

Which method or api is sending touches.
is touches sent to same that connect adb?
Which request body the scrcpy jar accepts for touch or keyboard commands.

Thank you

[drauggres]

Which method or api is sending touches.

#56

is touches sent to same that connect adb?

By default ws-scrcpy connects to scrcpy-server directly, without adb (unless proxy over adb was selected). For connection we use Web Sockets. Websockets are bidirectional, we receive video stream (and also video settings, clipboard) and send events.

Which request body the scrcpy jar accepts for touch or keyboard commands.

Same answer as in #56:

All classes for ControlMessages are here: https://github.com/NetrisTV/ws-scrcpy/tree/master/src/app/controlMessage.
For messages format look at the toBuffer method.

On server side you should look here and here.

[krishtoautomate]

if i use proxy over adb.
how touches are video buffer are communicated between device and computer?

[drauggres]

1. browser connects to the node.js server
2. server forwards port from device to localhost (same as adb forward tcp:<FREE_FORT> tcp:8886)
3. server creates proxy connection between browser and forwarded port

ws-scrcpy/src/server/goog-device/mw/WebsocketProxyOverAdb.ts

Lines 49 to 54 in 67cc51c

	public static createProxyOverAdb(ws: WebSocket, udid: string, remote: string, path?: string): WebsocketProxy {
	const service = new WebsocketProxy(ws);
	AdbUtils.forward(udid, remote)
	.then((port) => {
	return service.init(`ws://127.0.0.1:${port}${path ? path : ''}`);
	})

The rest is the same as with a direct connection.