1642854 - (hdr-video-linux) HDR video support for Linux

Status: UNCONFIRMED

(Core :: Audio/Video: Playback, enhancement)

Description

[romulasry]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0

Steps to reproduce:

Go to https://www.youtube.com/channel/UCve7_yAZHFNipzeAGBI5t9g/ and view a video on Linux.

Actual results:

VIdeos were in SDR

Expected results:

Been in HDR

Comment 1

[romulasry]

(In reply to romulasry from comment #0)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0

Steps to reproduce:

Go to https://www.youtube.com/channel/UCve7_yAZHFNipzeAGBI5t9g/ and view a video on Linux.

Actual results:

VIdeos were in SDR

Expected results:

Been in HDR

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 3

[Elle Solomina (she/her)]

Hey! It's still not working properly in Firefox 79 Oo Rly??? It's not a platform issue, on Windows and Linux, and Android is absolutely same, Firefox not working with HDR.

Comment 4

[Martin Stránský [:stransky] (ni? me)]

To have a roadmap here let's point out some tasks we need to finish to have HDR support on Linux:

• Extend existing NativeLayerWayland() to support external images (that's how video frames are injected to renderer). MacOS variant is here: https://phabricator.services.mozilla.com/D84638
• Extend DMABufSurfaces to hold HDR YUV surfaces (10-bit formats)
• Extend DMABufSurfaces to hold HDR color data
• Pass HDR data from FFmpeg decoder via DMABufSurfaces to MozContainer (or something else) and apply them during wl_surface_commit
• Propagate Linux/HDR support so web playes can serve us HDR content (i.e. clips in bt.2020 color space with yuv420p10 bit planes)
• Optionally update NativeLayerRootWayland to do local composition (by GL?) and send to Wayland compositor as less subsurfaces as possible.

Comment 5

[Martin Stránský [:stransky] (ni? me)]

Attachment: hdr-wip-2.patch

WIP v.2 patch, mostly contains unification for layer and multi buffer code.

Comment 6

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1642854 [Wayland] hdr patch

Comment 7

[Martin Stránský [:stransky] (ni? me)]

Attachment: WIP: Bug 1642854 [WIP][Wayland] hdr patch v.14 - viewport scale

Comment 8

[Martin Stránský [:stransky] (ni? me)]

A new patch which uses viewport scale instead of buffer scale. It should fix all the "wrong buffer size" bugs when Firefox is moved between screens with different scales. We can also remove all the stuff we use to stop compositor rendering during scale changes at nsWindow.

Comment 9

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1642854 [WIP][Wayland] hdr patch v.16

Comment 10

[Martin Stránský [:stransky] (ni? me)]

Attachment: WIP: Bug 1642854 [WIP][Wayland] hdr patch v.23

Comment 11

[Martin Stránský [:stransky] (ni? me)]

The patch here implements very experimental HDR playback. As it's missing correct wl_buffer format negotiation (Bug 1743631) and trows whatever it has to compositor it tends to crash due to unsupported modifiers. It needs further work on stabilization, layers rendering optimization and some layers fixes (D&D is broken due to missing layer screenshotter and there are rendering errors on scrolling).

Comment 12

[s.zharkoff]

I have tried 135beta1, 135beta2, trunk from yesterday morning - where fixes for wayland surfaces applied.

Everywhere notice a power consumption regression on idle. Same pages opened in 134 consumes 5 watts average on idle, 135 and trunk - 8 watts.
When playing videos on youtube the difference is not so big but still seems to be there - around 10 watts 134 and 11-12 135 and higher.

The FF processes in 135 and 134 seems to consume about the same CPU but with 135 version and higher compositor always runs at 10-13% CPU even when nothing is happening in FF. With 134 version at the same condition compositor is somewhere at the end of process list using abut 1%.

I understand that this is a very early WIP stage and such things is normal at this moment, just want to make early warning that there is something wrong in new wayland code related to the idle CPU/battery usage.

Use wayfire compositor based on wlroots 18.2.

Comment 13

[Martin Stránský [:stransky] (ni? me)]

(In reply to s.zharkoff from comment #12)

I have tried 135beta1, 135beta2, trunk from yesterday morning - where fixes for wayland surfaces applied.

Everywhere notice a power consumption regression on idle. Same pages opened in 134 consumes 5 watts average on idle, 135 and trunk - 8 watts.
When playing videos on youtube the difference is not so big but still seems to be there - around 10 watts 134 and 11-12 135 and higher.

The FF processes in 135 and 134 seems to consume about the same CPU but with 135 version and higher compositor always runs at 10-13% CPU even when nothing is happening in FF. With 134 version at the same condition compositor is somewhere at the end of process list using abut 1%.

I understand that this is a very early WIP stage and such things is normal at this moment, just want to make early warning that there is something wrong in new wayland code related to the idle CPU/battery usage.

Use wayfire compositor based on wlroots 18.2.

Do you mean with the WIP: Bug 1642854 [WIP][Wayland] hdr patch v.19 patch applied or without it?
Thanks.

Comment 14

[s.zharkoff]

Do you mean with the WIP: Bug 1642854 [WIP][Wayland] hdr patch v.19 patch applied or without it?
Thanks.

It was before applying this patch. As I understand this patch will not be in 135 - but 135 is already broken. Is this patch supposed to fix the idle issue?

wlroots is not yet capable of HDR signalling, so I do not expect to get HDR on my display and just view regular SDR videos, so in fact the issue is not because of sdr but because of some changes how ff talks with compositor, and those changes already landed in 135. So I guess my coments are not for thos patch but for the whole big wayland rework being done to allow HDR to happen.

Comment 15

[Martin Stránský [:stransky] (ni? me)]

(In reply to s.zharkoff from comment #14)

Do you mean with the WIP: Bug 1642854 [WIP][Wayland] hdr patch v.19 patch applied or without it?
Thanks.

It was before applying this patch. As I understand this patch will not be in 135 - but 135 is already broken. Is this patch supposed to fix the idle issue?

wlroots is not yet capable of HDR signalling, so I do not expect to get HDR on my display and just view regular SDR videos, so in fact the issue is not because of sdr but because of some changes how ff talks with compositor, and those changes already landed in 135. So I guess my coments are not for thos patch but for the whole big wayland rework being done to allow HDR to happen.

Okay, please file a new bug for it, I'll look at it. Please attach your about:support there.
Thanks.

Comment 16

[s.zharkoff]

Created the new bug
https://bugzilla.mozilla.org/show_bug.cgi?id=1940924

Comment 17

[Martin Stránský [:stransky] (ni? me)]

With the v.21 patch HDR is supposed to work. It includes dmabuf feedback and direct composition from external dmabuf surfaces (WebGL/VA-API). SW HDR rendering is not supported yet so you need VA-API decode of VP9 Profile 2 to decode P10 formats. It's under gfx.wayland.hdr pref.

Comment 18

[Martin Stránský [:stransky] (ni? me)]

There are still some bugs remaining - D&D doesn't work due to missing snapshotter and there are rendering artifacts like missing tiles during scrolling.

Comment 19

[Amine Hassane]

With the latest patch Firefox crashes for me upon starting a HDR video with this error:
GFX1-]: (kde) Wayland protocol error: zwp_linux_buffer_params_v1#857: error 7: importing the supplied dmabufs failed.

Comment 20

[Jonathan Bennett]

I'm also running into https://bugzilla.mozilla.org/show_bug.cgi?id=1732051 trying to test this. Once vsync is disabled, seeing the same error as Amine. Fedora 41 KDE, on an AMD card.

Comment 21

[Robert Mader [:rmader]]

FTR., for fully correct playback we'll also need to implement the color representation protocol - which is getting ready as well.

It mainly needed to tell the compositor whether to use BT601, BT709 or BT2020 matrix coefficients (and limited or full range) in the YCbCr->RGB conversion. Without that compositors like Mutter and KMS drivers will usually default to BT709 while one wants BT2020 for HDR10. The visual impact is not too bad, but visible once you compare directly next to each other.

See also my MR for Gstreamer as example.

Comment 22

[Martin Stránský [:stransky] (ni? me)]

Hello Robert,
we use xx_color_manager_v4 to set surface colorspace:
https://searchfox.org/mozilla-central/rev/d1fbe983fb7720f0a4aca0e748817af11c1a374e/widget/gtk/WaylandSurface.cpp#1356
isn't that enough?
Thanks.

Comment 23

[Robert Mader [:rmader]]

Only for RGB content like games. For YCbCr/YUV content, which is what we're aiming for here, you additionally need the matrix coefficients and range (and in theory chroma location) as specified in ITU-T Rec. 273 - i.e. for HDR10 you want the PQ curve (XX_COLOR_MANAGER_V4_TRANSFER_FUNCTION_ST2084_PQ) and the BT2020 primaries (XX_COLOR_MANAGER_V4_PRIMARIES_BT2020) from the color protocol in combination with BT2020 coefficients/limited range (WP_COLOR_REPRESENTATION_V1_COEFFICIENTS_BT2020/WP_COLOR_REPRESENTATION_V1_RANGE_LIMITED) from the color representation protocol.

But the color representation protocol won't make it into Gnome 48 anyways (I only added some initial plumbing for the KMS properties here) and the error is not too bad. Once the protocol lands - likely in 49 - it should be an easy addition. I just wanted to give you a heads-up.

Some context:

• Weston implementation https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1682

Comment 24

[wiagn233]

Final version of wayland color management protocol has just been merged: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/main/staging/color-management/color-management-v1.xml?ref_type=heads

Comment 25

[GreyXor]

Hello,
Just tried latest nightly and now it run correctly on sway with gfx.webrender.compositor.force-enabled
but I got crashes after randomly navigation

• https://crash-stats.mozilla.org/report/index/d2ee6e63-0cbf-43bc-ac5e-2ffe60250220
• https://crash-stats.mozilla.org/report/index/5cdc4aa3-2c69-4f77-a853-f478d0250220
• https://crash-stats.mozilla.org/report/index/24f593f3-adc5-4535-a5bf-26a780250220

Should I create new issues for that ?

Comment 26

[Martin Stránský [:stransky] (ni? me)]

(In reply to GreyXor from comment #25)

Hello,
Just tried latest nightly and now it run correctly on sway with gfx.webrender.compositor.force-enabled
but I got crashes after randomly navigation

• https://crash-stats.mozilla.org/report/index/d2ee6e63-0cbf-43bc-ac5e-2ffe60250220
• https://crash-stats.mozilla.org/report/index/5cdc4aa3-2c69-4f77-a853-f478d0250220
• https://crash-stats.mozilla.org/report/index/24f593f3-adc5-4535-a5bf-26a780250220

Should I create new issues for that ?

Yes please, you can file as blocking this one.

Comment 27

[wiagn233]

Btw we still have some color management issues even in SDR mode, which really needs paying attention on:
https://bugzilla.mozilla.org/show_bug.cgi?id=1939259
https://bugzilla.mozilla.org/show_bug.cgi?id=1834978

Comment 28

[GreyXor]

Latest build : https://hg.mozilla.org/mozilla-central/rev/54c1adbab9fb0376a3a0521762af1e52070bb2a5
Cannot see video anymore

tested https://download.blender.org/peach/bigbuckbunny_movies/BigBuckBunny_320x180.mp4
I can hear the sound, but the image remains black
with gfx.webrender.compositor.force-enabled

Comment 29

[GreyXor]

There a regression with VRR with compositor.force-enabled VRR is not working anymore.

I'm using vbltest to check the actual real refresh rate of my output.

Doing nothing: 32fps
Open nautilus and doing nothing: 32fps
Moving mouse anywhere: 60fps
Open firefox: 60fps
doing nothing in firefox: 60fps
open firefox (without compositor.fore-enabled): 32fps

If I disable compositor.force-enabled then vrr is working as expected again

Comment 30

[Martin Stránský [:stransky] (ni? me)]

(In reply to GreyXor from comment #29)

There a regression with VRR with compositor.force-enabled VRR is not working anymore.

I'm using vbltest to check the actual real refresh rate of my output.

Doing nothing: 32fps
Open nautilus and doing nothing: 32fps
Moving mouse anywhere: 60fps
Open firefox: 60fps
doing nothing in firefox: 60fps
open firefox (without compositor.fore-enabled): 32fps

If I disable compositor.force-enabled then vrr is working as expected again

That's possible, please file a new bug for it.
Thanks.