Conversion to RGB in VideoFrame.copyTo()

[Djuffin]

こんにちは TAG-さん!

I'm requesting a TAG review of Conversion to RGB in VideoFrame.copyTo().

VideoFrame.copyTo() can convert pixel data to RGB pixel format

Converting YUV video frames to RGB is often required for processing them in libraries like TensorFlow.js and OpenCV.js.
Previously the only possible way to achieve this was rendering the frame on a canvas.

Specifying VideoFrameCopyToOptions.format and VideoFrameCopyToOptions.colorSpace make it possible to asynchronously convert frames to RGB pixel format by calling VideoFrame.copyTo() without having to use an extra canvas.

• Explainer¹ (minimally containing user needs and example code): https://gist.github.com/Djuffin/9e2f98025ead49998524510cfeed8d33

• Specification URL: https://www.w3.org/TR/webcodecs/#dom-videoframecopytooptions-format

• Tests: https://github.com/web-platform-tests/wpt/blob/master/webcodecs/videoFrame-copyTo-rgb.any.js

• User research: N/A

• Security and Privacy self-review²: https://gist.github.com/Djuffin/c3742404b7c53ada227849c8b2b76b4c#file-webcodecs_contenthint_security_and_privacy-md

• Primary contacts (and their relationship to the specification):

  • Eugene Zemtsov ( @Djuffin ) Google LLC

• Organization(s)/project(s) driving the specification: Chromium

• Key pieces of existing multi-stakeholder (e.g. developers, implementers, civil society) support, review or discussion of this specification:

  • Mozilla comments: Allow converting VideoFrame to RGB pixel formats by calling VideoFrame.copyTo() mozilla/standards-positions#1017
  • WebKit comments: Allow converting VideoFrame to RGB pixel formats by calling VideoFrame.copyTo() WebKit/standards-positions#341

• External status/issue trackers for this specification (publicly visible, e.g. Chrome Status):

  • https://chromestatus.com/feature/4668827056209920

Further details:

• I have reviewed the TAG's Web Platform Design Principles
• Relevant time constraints or deadlines: couple of weeks
• The group where the work on this specification is currently being done: Media Working Group
• This work is being funded by: Google LLC