Presentation of objects

[mfoltzgoogle]

This is related to Issue #347. It would be ergonomic to allow objects accessible to the developer to be presentable on other devices via object URLs or source objects. This would simplify integration with Media Capture from Element.

Possible objects include:

• MediaStream
• MediaSource
• Blob
• File

The first two could be handled through the Remote Playback API (via HTMLMediaElement.srcObject).
It's just complicated plumbing to indirect a media stream through another element before sending it out of the browser.

The latter two can be sent across a PresentationConnection, but again, it would be simpler for the developer and the implementer to just send the raw object instead of constructing a document/connection for that purpose.

[anssiko]

I agree this sounds like a good API ergonomics improvement to explore.

I think it would be helpful to provide couple of concrete use cases along with JS code snippets to demonstrate the ergonomics advantage this would have.

[mfoltzgoogle]

To help organize the agenda, I'm marking a priority for this. It's worth exploring, but not a high priority use case for us.

[mfoltzgoogle]

It has been a very long time since I filed this issue, so I don't have any recollection of the original motivation or use cases. However let's assume that a Web application has access some local media it wants to play on a secondary display, and it's not hosted on a server (so it can't just send a URL or have the receiver page fetch it from elsewhere.)

If it's pre-recorded, encoded media (the MediaSource, Blob, or File cases), there are a few approaches to explore:

1. Via Presentation API: it could be sent in chunks as messages over the PresentationConnection, reassembled on the other side, and played out in an <audio>/<video> element.
2. Via Remote Playback API: it could be played out in a <video> element (possibly hidden) and the Remote Playback API used to play it out on the secondary display using API's "remoting" mode [*].

If it's captured media (the MediaStream case), the application could use WebCodecs to transform it into encoded media, then one of the two scenarios above used to take it from there.

It would be useful to prototype these scenarios with the current implementations of the APIs and see if there are any fundamental gaps in the platform, before pursuing any new API features. This would a good student project, or for anyone wanting to learn more about media processing on the Web platform.

There are also potential ways that WebRTC could be mixed into scenario #1. The PresentationConnection can serve as a signaling channel to set up the media plane between the controller and receiver. Then WebRTC becomes available for encoding/transmitting the media, with success depending on the capabilities of the receiver device.

[*] Chrome supports media remoting, but it is somewhat tricky to activate in its current form. Please reach out to me if you would like to investigate this further.