Adding support for High Dynamic Range (HDR) imagery to HTML Canvas

[palemieux]

こんにちは TAG-さん!

The Color on the Web CG is kindly requesting a TAG review of its proposed modification to the Web Platform to support High Dynamic Range (HDR) imagery in HTML Canvas.

The complete proposal is at https://github.com/w3c/ColorWeb-CG/blob/hdr_canvas_r2/hdr_html_canvas_element.md

Today HTML Canvas supports only 8 bit per color channel and two PredefinedColorSpace color spaces (srgb and display-p3). This is insufficient for High-Dynamic Range (HDR) imagery, which is in widespread use today. Accordingly, the following API modifications are needed to manipulate HDR images in HTML Canvas:

• add BT.2100 color spaces to PredefinedColorSpace
• add higher bit depth capabilities to CanvasRenderingContext2DSettings
• add higher bit depth capabilities to ImageDataSettings
• add image color volume information to ImageDataSettings and CanvasRenderingContext2DSettings
• add display color volume information to the Screen interface of the CSS Object Model, to determine the characteristics of the display on which the image is being reproduced

Please note that the proposed screenColorInfo property adds a fingerprinting surface. This feature is however intended primarily for advanced color volume mapping applications and is not believed to be generally necessary.

Contacts: @palemieux and @svgeesus (Color on the Web CG co-chairs)

Feedback, issues and questions are welcome at mailto:public-colorweb@w3.org and https://github.com/w3c/ColorWeb-CG/issues.

[svgeesus]

Hello TAG,

On 19 Feb 2024 in Tokyo there is a cross-SDO meeting HDR Stills Experts Day, with the International Color Consortium, W3C, and ISO TC 42/WG 23 (Photography) on the subject of HDR imagery. It would be super-useful to get any TAG input from this HDR in Canvas review ahead of that meeting

[palemieux]

@svgeesus Where is the meeting announced in the context of W3C? Any requirements for participation?

[svgeesus]

@palemieux It is on the ICC site in the Member-only area. It isn't a public meeting. I will be there, and Chris Cameron also.

[palemieux]

What is the process if myself and other W3C members wish to attend?

[palemieux]

To be clear, my concern is with the output of the meeting being described as including an opinion from the W3C and/or input provided in the meeting be presented as W3C input, without such input having been discussed/reviewed by W3C participants.

[svgeesus]

I think the proposed modification to the Web Platform to support High Dynamic Range (HDR) imagery in HTML Canvas has been adequately discussed that it can be presented at the meeting, no?

[palemieux]

Ok. Which document(s) are you planning on presenting -- beyond https://github.com/w3c/ColorWeb-CG/blob/hdr_canvas_r2/hdr_html_canvas_element.md? It would be good to inform their respective groups.

[jyasskin]

This doesn't seem like the right repository for this side discussion.

[LeaVerou]

Hi there,

@plinss @ylafon and I looked at this today during a breakout.
We had a few thoughts and questions:

1. The part of the API around checking whether a color is in within the volume of the output device and mapping it manually by the author seemed both too close to the hardware to be future-compatible, and too low level to have good DX. We think a higher level method to check whether a color is within the range of the output device would be preferable.
2. The Canvas APIs do not seem like the right place to specify primitives for querying the volume of the output device, which are useful in APIs across the entire web platform.
3. While we appreciated the discussion on volume mapping, we still had questions about the specifics. We see several cases where gamut/volume/tone mapping come into play, and we’d love some more detail on how these are handled (actually, the same questions apply to wider/smaller gamuts too):
   1. HDR image painted on a non-HDR canvas
   2. HDR canvas displayed non-HDR screen
   3. Author painting an HDR CSS color on a non-HDR canvas
4. The used color space is a property of CanvasRenderingContext2D, but HTMLCanvasElement provides methods for exporting the canvas contents as an image (toBlob(), toDataURL()). Is the color space preserved during these operations? (again this also applies to wide gamut)

And some more minor comments:

• Since CSS Color HDR has been approved as a workstream by the CSS WG, this work should probably reference it in the same way it references css-color-4.
• screen.screenColorInfo seems a little repetitive, why not screen.colorInfo?

[plinss]

I was also wondering why not include the other color spaces supported by CSS. I understand that the rec2100-* colorspaces are supersets, by if one was working in one of the smaller colorspaces the conversions to/from rec2011-* could be lossy, right?

I want to be sure that use cases aside from rendering the canvas directly to the screen are supported, so operations shouldn't be constrained by the available hardware. e.g. an image editor should be able to work natively in any source image's colorspace.

[svgeesus]

@plinss

I was also wondering why not include the other color spaces supported by CSS. I understand that the rec2100-* colorspaces are supersets, by if one was working in one of the smaller colorspaces the conversions to/from rec2011-* could be lossy, right?

Canvas 2D already supports any of those CSS Color 4 SDR color spaces on input. Currently, the only Canvas 2D context color spaces available are sRGB and Display P3 and the specified color is converted to the context colorspace. Furthermore, current implementations are restricted in practice to an 8-bit per component range:

Both CanvasRenderingContext2D and OffscreenCanvasRenderingContext2D contain an output bitmap that they render to. The pixel format of this output bitmap is currently unspecified. Many implementations use a 8 bits per channel RGB or RGBA pixel format for this bitmap.

Thus, if the specified color is outside the gamut of the 2D context, it cannot be represented (current implementations use a naive clamping operation to force the color to fit) and if it is in gamut, it will still be quantized to 8 bit resolution.

The HDR canvas proposal assumes as prerequisite the Canvas Floating Point Color Values proposal which introduces the ability to use floating-point pixel formats in CanvasRenderingContext2D, OffscreenCanvasRenderingContext2D, and ImageData.

Together with extended range colorspaces (the colorspace is defined over the entire coordinate range, i.e. not just [0.0, 1.0] but also negative numbers and numbers greater than 1) this would allow out of gamut input colors to be stored without loss. I don't know offhand if this is required and if only HDR colorspaces benefit from Canvas FP values, but I would assume that SDR colorspaces can utilize this.

[svgeesus]

I should clarify that all the CSS Color 4 color spaces are defined over the extended range.

[svgeesus]

@ palemieux this issue is waiting for a response from ColorWeb CG and/or the proposers, regarding the questions from the TAG review