Feasibility of using NymphCast as a wireless display for extending a computer's desktop

[candeira]

Microsoft has something called a Wireless display adapter, which I think is both a hardware product [1] and a software service integrated into Windows 10; the protocol is called Miracast. This not only casts media, but also allows the extension of the desktop [2] as though the remote screen were a hardware monitor connected via a cable to the desktop host's graphics adapter.

[1] https://www.amazon.com/Microsoft-P3Q-00001-Wireless-Display-Adapter/dp/B01AZC3J3M
[2] https://www.hanselman.com/blog/UseASecondLaptopAsAnExtendedMonitorWithWindows10WirelessDisplays.aspx

I'm asking this as an issue because the title is posed as question, but I would also like to convert it into a conversation on design and implementation of such a feature. I don't know any C++ and only userspace Linux programming, but I would be happy to help in the implementation of such a feature in any other way I can. Or maybe learn C++ and Linux kernel programming, who knows.

So, the two questions:

• Have you considered the use of of a networked display for desktop extension when designing NymphCast?
• If you haven't considered it, what do you think someone implementing this feature should do? Rather, what kind of implementation design for such a feature do you reckon you'd accept Pull Requests for?

[MayaPosch]

Remote display is a feature that I have planned to integrate into NymphCast at some point. Here I see the main difficulty being the actual capturing on the client's side. This becomes apparent when one starts looking into protocols like VNC and RDP. This would mostly target display duplication.

Ideally, one would like it to be a low-latency implementation, without a massive overhead. Sending screenshots in the case of display duplication is obviously a wasteful way to go about it, and doesn't solve the 'extending desktop' requirement.

For desktop extending, the first step would be to figure out how to implement this in a somewhat cross-platform fashion. Miracast support seems a bit spotty (Windows & Linux, no Mac, iOS or Android) and from what I read it's somewhat laggy and clunky. Maybe a better approach exists here or can be implemented?

[iopq]

@MayaPosch this is actually what I hoped I could do with NymphCast:

It should at first just do the display duplication, where it sends the live video of the display via display capture. But another feature is to simulate a virtual second monitor and duplicate that, if possible. The last step would be to enable a user to log in and create a graphical display without even being connected to a display in the first place.

Duplicating screen via Chrome works, but I can't do that programmatically via SSH so I can't just duplicate the screen without connecting a screen and clicking cast inside of Chrome, which is lame.

[Arndorferd]

This is an important feature, for some idea here, https://github.com/ASHS-School/mirrorcast, this is an open source project that has this in some form, perhaps looking at that would help. What seems important about this is as well, for video casting DRM implementation may be difficult as shown by the failure of matchstick and this could be used to bypass that as well?

[philtweir]

To simplify capturing, is this something that might theoretically be added as a provider to gnome-network-displays? The only current providers are Miracast / Wifi Direct and a dummy provider but they seem to have left a pretty general level of abstraction for other sinks, while handling the actual client side sources and sharing user flow (IIUC!) I guess that doesn't really answer the protocol question, or the cross-platform support, but perhaps this could decouple the sharing problem, which seems likely to be platform-specific, from the the sending implementation?

[SvenDowideit]

another related project may be https://github.com/albfan/miraclecast