1977744 - [HDR] WebRender HDR Rec2100PQ P010 video processing

Status: NEW

(Core :: Graphics: WebRender, task, P1)

Description

[Ashley Hale [:ahale]]

For universal support of HDR video in WebRender it will need to have appropriate shaders for converting colorspaces and pixel formats and transfer functions for the non-promoted surface case, and in some limited cases of conversion for promoted surfaces (e.g. Rec2100PQ colorspace with P010 pixel format is a YUV 4:2:0 format that will need to be converted to Rec2100PQ in RGB10A2 format to be put on an overlay in a lot of cases, and support for tonemapping on that is important, so that's a promoted surface case that needs these conversions), tonemapping for displays with less headroom than the content's maximum brightness is also necessary.

This roughly breaks down to a few stages of color conversion which can be performed in a shader:

• decode transfer function (PQ to linear, HLG to linear, or sRGB to linear)
• color matrix (to convert between colorspaces such as Rec2100 YUV, Rec2100 RGB, displayP3 RGB, sRGB, and Rec601 to generalize this for non-HDR video)
• tonemapping (e.g. using Reinhard tonemapping to adjust the content to fit a lower headroom than it was authored with, if the display has lower headroom, such as 4000 nits HDR on a DisplayHDR400 display which is 400 nits, or an 80 nits SDR display)
• encode transfer function (e.g. linear to PQ or HLG or sRGB)

Some example transformations that are expected:

• Rec2100PQ YUV in P010 format (a YUV 4:2:0 format) to Rec2100PQ RGB in RGB10A2 format (generally called HDR10), with tonemapping (e.g. fitting a 1000 nits video to a DisplayHDR400 nits display) - this is the most probable conversion we will be doing on Windows with an HDR display
• Rec2100HLG YUV in P010 format to Rec2100PQ RGB in RGB10A2 format
• Rec2100PQ YUV in P010 format to Rec2020 RGB in RGB10A2 format (i.e. DeepColor but not HDR)
• Rec2100PQ YUV in P010 format to sRGB in RGB10A2 format (again DeepColor but not HDR)
• Rec2100PQ YUV in P010 format to sRGB in BGRA8 format (this is our current approach to displaying SDR video)
• Rec2100PQ YUV in P010 format to Rec2100PQ YUV in P010 format with tonemapping (i.e. we're converting from YUV, converting to linear, passing through a no-op color matrix, applying tonemapping, and converting back to YUV) - for fullscreen video on Windows this may save power, other browsers favor this format for fullscreen video specifically so there is probably a good reason behind that, and it may be necessary to use this format when displaying on a laptop display that has HDR video but not HDR desktop support (those exist, believe it or not)
• Rec2100PQ YUV in P010 format to scRGB in RGBA16F format - it's not clear how often we'll have a use for this, but it is another way we can put video on the screen on Windows, and supprots transparency
• Rec2100PQ YUV in P010 format to Rec2020-extended RGB in RGBA16F format - this is a custom format we're exploring for rendering the web when HDR content is present, as it would preserve identical blending behavior for CSS/SVG purposes. No immediate uses for this format but it would make sense to support.

It's probable that we'll find other formats we want to support, which will be easy with the approach described above.

Feel free to ask ahale@ for explanation of any of the math involved.