1645671 - [Wayland] dmabuf-video-textures: Video is green the moment before video decoding starts

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect)

Description

[Darkspirit]

Attachment: Bildschirmfoto vom 2020-06-14 12-57-09.png

This is a minor cosmetic issue. It affects software and VAAPI.
It occurs only once. To reproduce again, reload the tab or change video resolution on YouTube.
(With media.ffvpx.enabled:false this can be observed with VP9 videos.)

software: MOZ_LOG="PlatformDecoderModule:5" MOZ_ENABLE_WAYLAND=1 mozregression --launch 0c7df6f9b0c1 --pref gfx.webrender.all:true widget.wayland-dmabuf-video-textures.enabled:true media.ffvpx.enabled:false -a https://bug1619882.bmoattachments.org/attachment.cgi?id=9149605 -P stdout

hardware: MOZ_LOG="PlatformDecoderModule:5" LIBVA_DRIVER_NAME=i965 MOZ_ENABLE_WAYLAND=1 mozregression --launch 0c7df6f9b0c1 --pref gfx.webrender.all:true widget.wayland-dmabuf-vaapi.enabled:true widget.wayland-dmabuf-video-textures.enabled:true media.ffvpx.enabled:false -a https://bug1619882.bmoattachments.org/attachment.cgi?id=9149605 -P stdout

Comment 1

[Marko]

I can confirm this bug, started happening a few days ago.

Comment 2

[ganlu]

I meet the similar issue with huya.com which is a live streaming website.

Comment 3

[Martin Stránský [:stransky] (ni? me)]

So this is a regression from Bug 1629788, correct?

Comment 4

[Darkspirit]

Correct.
https://hg.mozilla.org/integration/autoland/shortlog/0c7df6f9b0c1
first affected build: MOZ_LOG="PlatformDecoderModule:5" MOZ_ENABLE_WAYLAND=1 mozregression --repo autoland --launch 0c7df6f9b0c1 --pref gfx.webrender.all:true widget.wayland-dmabuf-video-textures.enabled:true media.ffvpx.enabled:false -a https://bug1619882.bmoattachments.org/attachment.cgi?id=9149605 -P stdout
last unaffected build: MOZ_LOG="PlatformDecoderModule:5" MOZ_ENABLE_WAYLAND=1 mozregression --repo autoland --launch 6f6757bfd3298d8314d52730c3c016dc4c629f05 --pref gfx.webrender.all:true widget.wayland-dmabuf-video-textures.enabled:true media.ffvpx.enabled:false -a https://bug1619882.bmoattachments.org/attachment.cgi?id=9149605 -P stdout

Comment 5

[Darkspirit]

Same for VAAPI:
first affected build: MOZ_LOG="PlatformDecoderModule:5" LIBVA_DRIVER_NAME=i965 MOZ_ENABLE_WAYLAND=1 mozregression --launch 0c7df6f9b0c1 --pref gfx.webrender.all:true widget.wayland-dmabuf-vaapi.enabled:true widget.wayland-dmabuf-video-textures.enabled:true media.ffvpx.enabled:false -a https://bug1619882.bmoattachments.org/attachment.cgi?id=9149605 -P stdout
last unaffected build: MOZ_LOG="PlatformDecoderModule:5" LIBVA_DRIVER_NAME=i965 MOZ_ENABLE_WAYLAND=1 mozregression --repo autoland --launch 6f6757bfd3298d8314d52730c3c016dc4c629f05 --pref gfx.webrender.all:true widget.wayland-dmabuf-vaapi.enabled:true widget.wayland-dmabuf-video-textures.enabled:true media.ffvpx.enabled:false -a https://bug1619882.bmoattachments.org/attachment.cgi?id=9149605 -P stdout

Comment 6

[Marko]

This started happening again after today's update...

Comment 7

[Darkspirit]

This was never gone and the feature is not finished yet (bug 1629788 comment 27). Some other patches still await review.
VAAPI was just temporarily disabled: bug 1645706 disabled the dmabuf-video-textures pref that VAAPI now depends upon. You had to enable two prefs for VAAPI. bug 1646051 changed it to let the VAAPI pref also enable dmabuf-video-textures. You see this bug again because you are using VAAPI again.

Comment 8

[Marko]

(In reply to Jan Andre Ikenmeyer [:darkspirit] from comment #7)

This was never gone and the feature is not finished yet (bug 1629788 comment 27). Some other patches still await review.
VAAPI was just temporarily disabled: bug 1645706 disabled the dmabuf-video-textures pref that VAAPI now depends upon. You had to enable two prefs for VAAPI. bug 1646051 changed it to let the VAAPI pref also enable dmabuf-video-textures. You see this bug again because you are using VAAPI again.

Thanks for clarification! I thought it was gone for a few days.

Comment 9

[George Sofianos]

(In reply to Marko from comment #8)

Thanks for clarification! I thought it was gone for a few days.

I never had this on my PC (AMD 5700XT / mesa 20.1.1 / Linux 5.7.2) and I was using VAAPI decoding for about a month. Just started happening today. Even with yesterdays nightly where I had to enable the dmabuf-video-textures pref, I was seeing no Green frames. I'm just commenting it in case it helps to identify what it is, but since it's work in progress I guess it will be resolved at some point.

Comment 11

[Ilya K]

For reference, this is also an issue on X11/EGL, even without media.ffmpeg.dmabuf-textures.enabled.

Comment 12

[Marko]

Why is this bug of such low priority, it is a regression that basically isn't present on 78 stable and will regress both 79 and even 80 it seems for all the vaapi users... is there any way that fix will be backported once it's fixed?

Comment 13

[Mel]

I agree with comment 12, now that firefox 79 is officially out this bug is very visible to anyone using hardware accelerated video playback on linux.

Comment 14

[Robert Mader [:rmader]]

(In reply to Marko from comment #12)

Why is this bug of such low priority, it is a regression that basically isn't present on 78 stable and will regress both 79 and even 80 it seems for all the vaapi users... is there any way that fix will be backported once it's fixed?

First of all this feature is not yet enabled by default and requires users to jump through several hoops to enable it (enable EGL either for X11 or by using the Wayland backend, enable WR, enable several options). Secondly it's not a crash but just a glitch, and not a major one neither. Finally the feature was not developed by the FF core team but external contributors. This together results in a low priority. And no, fixes for disabled-by-default features will usually not get backported.

But don't worry, I'm sure Martin will work on it as time allows.

Comment 15

[Marko]

(In reply to Robert Mader [:rmader] from comment #14)

(In reply to Marko from comment #12)

Why is this bug of such low priority, it is a regression that basically isn't present on 78 stable and will regress both 79 and even 80 it seems for all the vaapi users... is there any way that fix will be backported once it's fixed?

First of all this feature is not yet enabled by default and requires users to jump through several hoops to enable it (enable EGL either for X11 or by using the Wayland backend, enable WR, enable several options). Secondly it's not a crash but just a glitch, and not a major one neither. Finally the feature was not developed by the FF core team but external contributors. This together results in a low priority. And no, fixes for disabled-by-default features will usually not get backported.

But don't worry, I'm sure Martin will work on it as time allows.

I know, but it's a regression... shouldn't regressions have higher priority than implementing new features like X11 backend etc... the part that is bad about this is that in 78 it works perfectly without any issues, and in 79+ is annoying to use again just like with frame skipping bug before, and yet there are new features being worked on without finishing old ones first.
Obviously it's up to Martin to decide since he is a contributor and FF devs don't care enough to provide tge feature, but as a linux laptop user, I would really like to see this having a higher priprity :) didn't mean to be rude or anything.

Comment 17

[Martin Stránský [:stransky] (ni? me)]

(In reply to Marko from comment #15)

I know, but it's a regression... shouldn't regressions have higher priority than implementing new features like X11 backend etc... the part that is bad about this is that in 78 it works perfectly without any issues, and in 79+ is annoying to use again just like with frame skipping bug before, and yet there are new features being worked on without finishing old ones first.
Obviously it's up to Martin to decide since he is a contributor and FF devs don't care enough to provide tge feature, but as a linux laptop user, I would really like to see this having a higher priprity :) didn't mean to be rude or anything.

The patch is on the way. I prefer to unify X11/Wayland VA-API/dmabuf backends and then do all the fixes for both together, that lowers the maintenance cost. This is really still an experimental feature so please accept potential breakage during the development (and I also have other duties at Red Hat).

Comment 18

[Marko]

(In reply to Martin Stránský [:stransky] from comment #17)

(In reply to Marko from comment #15)

I know, but it's a regression... shouldn't regressions have higher priority than implementing new features like X11 backend etc... the part that is bad about this is that in 78 it works perfectly without any issues, and in 79+ is annoying to use again just like with frame skipping bug before, and yet there are new features being worked on without finishing old ones first.
Obviously it's up to Martin to decide since he is a contributor and FF devs don't care enough to provide tge feature, but as a linux laptop user, I would really like to see this having a higher priprity :) didn't mean to be rude or anything.

The patch is on the way. I prefer to unify X11/Wayland VA-API/dmabuf backends and then do all the fixes for both together, that lowers the maintenance cost. This is really still an experimental feature so please accept potential breakage during the development (and I also have other duties at Red Hat).

Thnaks for the clarification about X11/wayland common backend.

Don't worry Martin, we owe you eternal gratitude for implementing this feature in Firefox after so many years the issue was neglected by other developers!

Comment 19

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1645671 [Linux/VA-API] Create DMABufSurfaceWrapper directly at nsTAttay and disable DMABufSurfaceWrapper class copy and assignment constructors, r?jya

When DMABufSurfaceWrapper is added to nsTArray, a temporary local DMABufSurfaceWrapper object is created. When the temporary
object is deleted after the adding, associated dmabuf data is released which leads to rendering artifact during VA-API video playback.

As a fix in this patch we use UniquePtr<DMABufSurfaceWrapper> at nsTArray thus we don't create the temporary DMABufSurfaceWrapper object.
Also change GetUnusedDMABufSurfaceWrapper() to GetUnusedDMABufSurfaceWrapperIndex() and reference the surface wrapper by indexe instead of pointer.

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bacacb384008
[Linux/VA-API] Create DMABufSurfaceWrapper directly at nsTAttay and disable DMABufSurfaceWrapper class copy and assignment constructors, r=jya

Comment 24

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/bacacb384008

Comment 25

[BugBot [:suhaib / :marco/ :calixte]]

Since the status are different for nightly and release, what's the status for beta?
For more information, please visit auto_nag documentation.

Comment 26

[Darkspirit]

[Tracking Requested - why for this release]:

This regression affects:

• MOZ_ENABLE_WAYLAND=1 users with Firefox 79,
• MOZ_X11_EGL=1 and MOZ_ENABLE_WAYLAND=1 users with Firefox 80.

Everyone is excited that Firefox 80 finally supports VAAPI hardware decoding on X11, it got news coverage, etc., but those who tested it know that it suffers from at least two fatal bugs. (bug 1645672 is the other one.) Wayland/MOZ_X11_EGL and VAAPI are still experimental and disabled-by-default, but it's a killer feature that brings users back. Therefore please consider uplifing this into beta, and even into a dot release, if there is any.

Downstream Fedora applies this patch on Firefox 79: https://src.fedoraproject.org/rpms/firefox/c/cd18e999f576523b6b3f2076bca625d5e04d1178?branch=master

https://www.reddit.com/r/archlinux/comments/i0ireu/firefox_developer_edition_hardware_video/fzpojad/
https://www.reddit.com/r/flatpak/comments/i157dm/green_flashes_in_video_in_firefox/
https://www.reddit.com/r/linux/comments/i04cvr/psa_firefox_79_appears_to_break_vaapi_decoding_on/
https://www.reddit.com/r/firefox/comments/i1p2ct/linux_vaapi_video_decode_acceleration_unreliable/

Comment 27

[Julien Cristau [:jcristau]]

Not tracking as this is still disabled.

Comment 28

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:stransky, is this bug important enough to require an uplift?
If not please set status_beta to wontfix.

For more information, please visit auto_nag documentation.

Comment 29

[Martin Stránský [:stransky] (ni? me)]

Comment on attachment 9166559 [details]
Bug 1645671 [Linux/VA-API] Create DMABufSurfaceWrapper directly at nsTAttay and disable DMABufSurfaceWrapper class copy and assignment constructors, r?jya

Beta/Release Uplift Approval Request

• User impact if declined: Broken VA-API video playback on Wayland/X11.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Changes a way how elements are stored in nsTArray, to in-place assignment.
• String changes made/needed:

Comment 30

[Julien Cristau [:jcristau]]

Comment on attachment 9166559 [details]
Bug 1645671 [Linux/VA-API] Create DMABufSurfaceWrapper directly at nsTAttay and disable DMABufSurfaceWrapper class copy and assignment constructors, r?jya

wayland/vaapi fix, approved for 80.0b5

Comment 31

[Julien Cristau [:jcristau]]

https://hg.mozilla.org/releases/mozilla-beta/rev/8d720959430b

Comment 32

[Ronan Jouchet [:ronjouch]]

Hi.

Trying VAAPI/X11 on Nightly 81.0a1/20200807213618, intel_gpu_top confirms HW acceleration is active, but all videos appears constantly green, for the whole duration of the video (not just for a moment or a few frames) . Is there already a bug for it, or should I file one?

• MOZ_X11_EGL=1
• media.ffmpeg.dmabuf-textures.enabled:true , media.ffmpeg.vaapi.enabled:true , media.ffvpx.enabled:false
• enhanced-h264ify forcing H264
• Arch, latest linux / xorg / intel drivers. Problem reproducible with both i965 and iHD drivers.

If already answered, sorry for the noise. If not, please tell me what debug info you'd like me to share in a new bug.

Comment 33

[Ronan Jouchet [:ronjouch]]

(In reply to Ronan Jouchet from comment #32)

Trying VAAPI/X11 on Nightly 81.0a1/20200807213618, intel_gpu_top confirms HW acceleration is active, but all videos appears constantly green, for the whole duration of the video (not just for a moment or a few frames) . Is there already a bug for it, or should I file one?

• MOZ_X11_EGL=1
• media.ffmpeg.dmabuf-textures.enabled:true , media.ffmpeg.vaapi.enabled:true , media.ffvpx.enabled:false
• enhanced-h264ify forcing H264
• Arch, latest linux / xorg / intel drivers. Problem reproducible with both i965 and iHD drivers.

Additional info: issue reproduced on a fresh Nightly profile with zero config/extensions except for the above prefs, on a mp4/h264 file I'm sure my hardware can decode (according to vainfo), http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_30fps_normal.mp4 . I hear the sound, but video surface is completely green all the time.

Comment 34

[Anan Laks [:vulcansphere]]

(In reply to Ronan Jouchet from comment #33)

(In reply to Ronan Jouchet from comment #32)

Trying VAAPI/X11 on Nightly 81.0a1/20200807213618, intel_gpu_top confirms HW acceleration is active, but all videos appears constantly green, for the whole duration of the video (not just for a moment or a few frames) . Is there already a bug for it, or should I file one?

• MOZ_X11_EGL=1
• media.ffmpeg.dmabuf-textures.enabled:true , media.ffmpeg.vaapi.enabled:true , media.ffvpx.enabled:false
• enhanced-h264ify forcing H264
• Arch, latest linux / xorg / intel drivers. Problem reproducible with both i965 and iHD drivers.

Additional info: issue reproduced on a fresh Nightly profile with zero config/extensions except for the above prefs, on a mp4/h264 file I'm sure my hardware can decode (according to vainfo), http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_30fps_normal.mp4 . I hear the sound, but video surface is completely green all the time.

I can reproduce this on my system with AMDGPU driver and with same about:config entries as you.

Comment 35

[Anan Laks [:vulcansphere]]

(In reply to Anan Laks from comment #34)

(In reply to Ronan Jouchet from comment #33)

(In reply to Ronan Jouchet from comment #32)

Trying VAAPI/X11 on Nightly 81.0a1/20200807213618, intel_gpu_top confirms HW acceleration is active, but all videos appears constantly green, for the whole duration of the video (not just for a moment or a few frames) . Is there already a bug for it, or should I file one?

• MOZ_X11_EGL=1
• media.ffmpeg.dmabuf-textures.enabled:true , media.ffmpeg.vaapi.enabled:true , media.ffvpx.enabled:false
• enhanced-h264ify forcing H264
• Arch, latest linux / xorg / intel drivers. Problem reproducible with both i965 and iHD drivers.

Additional info: issue reproduced on a fresh Nightly profile with zero config/extensions except for the above prefs, on a mp4/h264 file I'm sure my hardware can decode (according to vainfo), http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_30fps_normal.mp4 . I hear the sound, but video surface is completely green all the time.

I can reproduce this on my system with AMDGPU driver and with same about:config entries as you.

With both H264 and VP9 video.

Comment 36

[u658943]

I am experiencing the exact same issue as Ronan Jouchet and Anan Laks.

• I am on X11/intel modesetting
• I have MOZ_X11_EGL=1, media.ffmpeg.dmabuf-textures.enabled:true , media.ffmpeg.vaapi.enabled:true , media.ffvpx.enabled:false
• I play a video, check with intel_gpu_top that HW acceleration is working (it is)
• the video is just green.

Like Ronan I have also been able to reproduce this on a completely vanilla nightly profile. This happens with both h264 and VP9 video like in Anan's case.

Comment 37

[Robert Mader [:rmader]]

It's likely that you see bug 1658035 - should be fixed by next nightly.

Comment 38

[Martin Stránský [:stransky] (ni? me)]

Please file a new bug about it and also attach a media log.
Run Firefox on terminal with MOZ_LOG="PlatformDecoderModule:5" to get the media log.
My guess is that we don't import dmabuf surfaces correctly, the decoder may use a different format or so.
Thanks.

Comment 39

[Darkspirit]

Yes, bug 1658035 is about videos being completely green until the next Nightly update.

Comment 40

[Ronan Jouchet [:ronjouch]]

Well that was fast :D . Fix confirmed in today's Nightly 81.0a1/20200808093545, VAAPI acceleration is effective for me under Xorg/Intel, and videos are no longer completely green. Thanks for the fix, Robert, and thanks for the Xorg push, Martin!

With that, not creating a new bug. If you nevertheless still would like the info you requested in Comment 38, Martin, do ping me and I'll create one.

Comment 41

[Anan Laks [:vulcansphere]]

(In reply to Ronan Jouchet from comment #40)

Well that was fast :D . Fix confirmed in today's Nightly 81.0a1/20200808093545, VAAPI acceleration is effective for me under Xorg/Intel, and videos are no longer completely green. Thanks for the fix, Robert, and thanks for the Xorg push, Martin!

With that, not creating a new bug. If you nevertheless still would like the info you requested in Comment 38, Martin, do ping me and I'll create one.

Can confirm, now VAAPI acceleration is working properly on latest Nightly build.

Thanks for the fix, everyone.

Comment 42

[susenoaditya]

This is happened to me when widget.wayland-dmabuf-vaapi.enabled is set to True