Closed Bug 1802357 Opened 2 years ago Closed 11 months ago

Hardware accelerated video sometimes stops working

Categories

(Core :: Audio/Video: Playback, defect)

defect

Tracking

()

RESOLVED FIXED
116 Branch
Tracking Status
firefox116 --- fixed

People

(Reporter: jrmuizel, Assigned: sotaro)

Details

Attachments

(10 files, 1 obsolete file)

See https://www.reddit.com/r/firefox/comments/x3fz1x/youtube_videos_failing_to_start_playing/itex9qj/

We should try to put something in gfx critical log in about:support if this happens.

John, can you think of how we can debug this?

Flags: needinfo?(jolin)

hi guys. have you been able to make any progress on this? the bug seems to be reappearing more frequently lately... it's been around for so long at this point. I would really appreciate a fix or some sign that it can be detected and replicated at the very least

hi again, guys. any ideas on this one yet?
I've since tried refreshing my Firefox instance but that didn't help. then I tried reinstalling the browser and that looked like it was gonna help as I didn't see any videos failing to play for a while but eventually, AVC videos started to break for me

can someone please look into this one?

(In reply to housebottle from comment #3)

hi again, guys. any ideas on this one yet?
I've since tried refreshing my Firefox instance but that didn't help. then I tried reinstalling the browser and that looked like it was gonna help as I didn't see any videos failing to play for a while but eventually, AVC videos started to break for me

can someone please look into this one?

Hi,

So sorry to hear that it still happens to you. Could you please help attach the whole about:support raw data here again? Also, please upload the modified preferences from about:config and the console log from the web developer tools for us to investigate.

Thanks a lot!

Flags: needinfo?(jolin) → needinfo?(eheo4trlh)
Attached file modified config
thanks for the reply,  John

attached
Attached file about:support

and did you mean the console log for this page or for when the video fails to play? if it's the latter, I'll post it the next time it occurs.

sorry for posting a separate comment for each attachment. IDK how to attach multiple files in the same comment, if that's even possible

Flags: needinfo?(eheo4trlh)

hi Jim,

so v.redd.it videos have stopped working for me again. and this is the console output:

13:19:35.248 Referrer Policy: Ignoring the less restricted referrer policy “unsafe-url” for the cross-site request: https://www.redditstatic.com/crossposting.4zJErPF9qdo.js crossposting.4zJErPF9qdo.js
13:19:35.248 Referrer Policy: Ignoring the less restricted referrer policy “unsafe-url” for the cross-site request: https://www.redditstatic.com/videoplayer.pRlfb8S7mb4.js videoplayer.pRlfb8S7mb4.js
13:19:35.249 Referrer Policy: Ignoring the less restricted referrer policy “unsafe-url” for the cross-site request: https://a.thumbs.redditmedia.com/R_T223YbptDrO5V9OXyW4Htd4i82RtgfF0234qvnIO0.jpg R_T223YbptDrO5V9OXyW4Htd4i82RtgfF0234qvnIO0.jpg
13:19:35.478 Referrer Policy: Ignoring the less restricted referrer policy “unsafe-url” for the cross-site request: https://www.redditstatic.com/gold/awards/icon/silver_48.png silver_48.png
13:19:35.478 Referrer Policy: Ignoring the less restricted referrer policy “unsafe-url” for the cross-site request: https://www.redditstatic.com/gold/awards/icon/gold_48.png gold_48.png
13:19:35.479 Referrer Policy: Ignoring the less restricted referrer policy “unsafe-url” for the cross-site request: https://preview.redd.it/award_images/t5_22cerq/klvxk1wggfd41_Helpful.png?width=48&height=48&auto=webp&s=e50064b090879e8a0b55e433f6ee61d5cb5fbe1d klvxk1wggfd41_Helpful.png
13:19:35.479 Referrer Policy: Ignoring the less restricted referrer policy “unsafe-url” for the cross-site request: https://www.redditstatic.com/gold/awards/icon/Starry_48.png Starry_48.png
13:19:35.737 Referrer Policy: Ignoring the less restricted referrer policy “unsafe-url” for the cross-site request: https://www.redditstatic.com/reddit.en.JhgeaWBdlAQ.js reddit.en.JhgeaWBdlAQ.js
13:19:35.737 Referrer Policy: Ignoring the less restricted referrer policy “unsafe-url” for the cross-site request: https://www.redditstatic.com/_chat.mLSe5kQYhig.js _chat.mLSe5kQYhig.js
13:19:35.737 Referrer Policy: Ignoring the less restricted referrer policy “unsafe-url” for the cross-site request: https://www.redditstatic.com/spoiler-text.vsLMfxcst1g.js spoiler-text.vsLMfxcst1g.js
13:19:35.737 Referrer Policy: Ignoring the less restricted referrer policy “unsafe-url” for the cross-site request: https://www.redditstatic.com/onetrust.6tPW2jUogoc.js 2 onetrust.6tPW2jUogoc.js
13:19:36.526 Loading failed for the <script> with source “https://www.redditstatic.com/onetrust.6tPW2jUogoc.js”. the_us_military_used_compressed_air_to_deliver:1310:1
13:19:36.726
Some cookies are misusing the recommended “SameSite“ attribute 2
13:19:38.362
                  ,d"=≥,.,qOp,
                 ,7'  ''²$(  )
                ,7'      '?q$7'
             ..,$$,.
   ,.  .,,--***²""²***--,,.  .,
 ²   ,p²''              ''²q,   ²
:  ,7'                      '7,  :
 ' $      ,db,      ,db,      $ '
  '$      ²$$²      ²$$²      $'    Using Reddit at work? Work for Reddit.
  '$                          $'      https://www.redditinc.com/careers
   '$.     .,        ,.     .$'
    'b,     '²«»«»«»²'     ,d'
     '²?bn,,          ,,nd?²'
       ,7$ ''²²²²²²²²'' $7,
     ,² ²$              $² ²,
     $  :$              $:  $
     $   $              $   $
     'b  q:            :p  d'
      '²«?$.          .$?»²'
         'b            d'
       ,²²'?,.      .,?'²²,
      ²==--≥²²==--==²²≤--==²
index.ts:5:10
13:19:42.840 Autoplay is only allowed when approved by the user, the site is activated by the user, or media is muted. foreground.entry.js:16148:34
13:19:42.841 Uncaught (in promise) DOMException: The play method is not allowed by the user agent or the platform in the current context, possibly because the user denied permission. 

does this console output in addition to the about:support and about:config attachments help you out or give you any additional information at all?

Flags: needinfo?(jmathies)
Attached file console output
and now AVC YouTube videos have stopped working. console output attached

13:19:42.840 Autoplay is only allowed when approved by the user, the site is activated by the user, or media is muted. foreground.entry.js:16148:34
13:19:42.841 Uncaught (in promise) DOMException: The play method is not allowed by the user agent or the platform in the current context, possibly because the user denied permission.

What do you have your autoplay settings set to?

Flags: needinfo?(jmathies)

Hi Jim,

I've attached the modified about:config above which contains the following lines:

media.autoplay.blocking_policy 2
media.autoplay.default 5

I believe I changed those to disable autoplay on YouTube based on some reddit threads on /r/firefox. My current autoplay settings have been set to "Block Audio and Video" as the default for all websites. This alone without the about:config modifications don't work for YouTube hence the modified settings...

Could this be what is causing the playback to fail? I could undo those changes but I strongly suspect the out-of-the-box autoplay settings on Firefox are not going to disable autoplay on YouTube.

Attached file console output
so I undid all the modified config related to autoplay and I also allowed autoplay in the settings via GUI. and it still fails to play some videos... here's the console output when I tried to play the video on this page: https://www.reddit.com/r/tennis/comments/1090wvk/the_flying_selfie/
Attached file console
so I undid all the modified config related to autoplay and I also allowed autoplay in the settings via GUI. and it still fails to play some videos... here's the console output when I tried to play the video on this page: https://www.reddit.com/r/tennis/comments/1090wvk/the_flying_selfie/

hope it helps
Attached file output
so I undid all the modified config related to autoplay and I also allowed autoplay in the settings via GUI. and it still fails to play some videos... here's the console output when I tried to play the video on this page: https://www.reddit.com/r/tennis/comments/1090wvk/the_flying_selfie/

hope it helps
Attached file modified about config
so I undid all the modified config related to autoplay and I also allowed autoplay in the settings via GUI. and it still fails to play some videos... here's the console output when I tried to play the video on this page: https://www.reddit.com/r/tennis/comments/1090wvk/the_flying_selfie/

https://pastebin.com/Yct5TKgh


does that tell you anything useful? I think I am just going to have to accept that disabling autoplay reliably is just something Firefox can't do... but videos are still breaking. to clean up my `about:config`, I checked the box to display only the modified changes and then searched "autoplay" and undid all the changes there. here's what the modified `about:config` looks like

yikes. sorry for the spam, guys. it kept telling me that it failed to attach in a pop-up window and the page didn't seem to have my posts so I had to use pastebin to paste that... but then when I refreshed, all the previous posts were already there... can you please delete the other posts except for the last one before this?

also, I just noticed that the about:config attachment I submitted on 2022-12-12 22:59 PST has some personally identifying information. can you please delete that as well? I hadn't realised it would have my email on there.

thanks. sorry again for the spam

Flags: needinfo?(alwu)
Attached file console output
hey guys, thanks for deleting the file... has any progress been made? I still constantly encounter videos that fail to load, especially when I'm browsing reddit and watching v.redd.it but it's not exclusive to reddit. tried to play a random video on vimeo https://vimeo.com/791431151 and it just won't play... same video plays flawlessly on Chrome. I don't understand why videos seem to be breaking for me and apparently nobody else. is it the add-ons interfering? some other setting? I tried to look at the console output too but I'm not sure if it says anything useful. just want to know what the root cause is...

Per comment 17, would you mind to do a profiling while the issue is happening on that vimeo video? Here is the tutorial video about how to profile firefox step by step, thank you.

Flags: needinfo?(alwu) → needinfo?(eheo4trlh)

(In reply to Alastor Wu [:alwu] from comment #18)

Per comment 17, would you mind to do a profiling while the issue is happening on that vimeo video? Here is the tutorial video about how to profile firefox step by step, thank you.

hi Alastor, sorry about the delay. I've just been waiting for the video playback to break again at the right time to profile. it has just broken and I went to about:logging but I don't see a "Logging preset" dropdown menu. I do have a textbox that lists the different modules though

Current Log Modules:
timestamp,sync,nsSocketTransport:5,nsHostResolver:5,cache2:5,nsHttp:5

I've just manually typed in the modules I saw in the video and appended it to the above modules so now the modules are:

Current Log Modules:
timestamp,sync,PlatformDecoderModule:5,nsSocketTransport:5,MediaDecoder:4,AudioSink:5,nsHostResolver:5,cache2:5,MediaDecoderStateMachine:4,AudioSinkWrapper:5,nsHttp:5

hope that's okay. here's the file: https://share.firefox.dev/3jogOUk

I played the video and then skipped ahead to a random point in the video when it didn't play. I think I did it correctly. please let me know if you need me to do it again.

thanks!

Flags: needinfo?(eheo4trlh)

hi team

any updates on this? I'm still seeing this bug pop up every now and then. would be fantastic if someone could identify the root cause. if it's something with the way my instance of Firefox is set up, I would love to tweak the config

please let me know

thanks

guys... this issue is still happening. is there any hint as to what might be happening? I am so frustrated... especially since most people don't seem to be experiencing it and we don't seem to be have found out why playback is breaking several months after first reporting this problem... I am willing to do a live conference call if that helps debug this

I found a thread on /r/firefox by a user who seems to be experiencing the same issue here: https://www.reddit.com/r/firefox/comments/11lqxe0/all_firefox_above_102_have_playback_issues_how_to/

please give me an update

can I get an update please? is anyone even looking at it at this point?

is it possible to setup a video call and debug this in person? I am willing to assist with that if it's possible

Flags: needinfo?(alwu)

Sorry for my late reply, feel free to NI (needinfo) me next time if I didn't response. Okay so I checked your profiled result in the comment 19, it's pretty weird, I can see you performed a lot of seeking, but none of them succeeded. It seems we didn't get any data at all.

Could you help me perform profiling again by following the same instructions, but this time I'd need you to add more log modules, please help me paste following timestamp,HTMLMediaElement:4,HTMLMediaElementEvents:4,PlatformDecoderModule:5,AudioSink:5,AudioSinkWrapper:5,MediaDecoderStateMachine:5,MediaDecoder:5,MediaFormatReader:5,MediaSource:5 into New log modules and then click set log module. After that, perform the profiling again.

Thank you so much!

Flags: needinfo?(alwu) → needinfo?(eheo4trlh)

Hi Alastor,

Thanks for the reply. I've been using Firefox with the hardware acceleration turned off and that has been working for me but everything's been noticeably slower because of it.

I also noticed that after upgrading to 113.0.0, I was able to leave the hardware acceleration turned on and use the browser just fine. Videos never broke for the 5 or 6 days that I used Firefox 113.0.0 for. After upgrading the browser to 113.0.1 and using it for about 10ish hours, the videos broke once again. It is possible that on 113.0.0, I just got lucky. Since I don't know how to replicate the bug, I just wait for the videos to break on their own. They never did on 113.0.0 but they did on 113.0.1. As always, restarting the browser fixes the problem.

Here's the profiler data: https://share.firefox.dev/42H0RK9. I could see the video loading in the progress bar but it didn't play.

Hope this helps.

Thanks

Flags: needinfo?(eheo4trlh) → needinfo?(alwu)

just want to add that it's definitely worse on 113.0.1. I went several days without any issues on 113.0.0 but with 113.0.1, the playback breaks very often, even with hard acceleration turned off :/

So sorry to hear that the issue still happens :( I'd like to confirm with you, the profiled result you provided in comment 25, was starting profiling after video stopped working right? And I saw you tried to seek it multiple times, but the video didn't respond at all.

From the profiled result, I saw that the decoder threads in the GPU process completely stuck. They seems stuck on the decoder deconstruction happened before (when video stopped working) They never finished, so we were not able to create new video decoder. I also noticed that (I should have noticed that earlier, sorry) your graphic card is HD 620, and we've received many hardware decoding issue on that graphic card before. Would you mind to try following instructions, to see if any of them help?

  1. update the driver for HD 620
    I saw your driver is from "10-14-2020", it could be a driver issue which they fix already.

  2. disable media.wmf.zero-copy-nv12-textures

Thank you so much!

Flags: needinfo?(alwu) → needinfo?(eheo4trlh)

Hi Alastor,

Thanks for your reply. Yes, I did start the profiler after I noticed that the playback stopped working again. So I visited a video on Vimeo and then hit play. And when it didn't load, I tried to seek to random points in the video. That didn't load either after which I stopped the profiling.

As far as updating the driver goes, I visited this page: https://www.intel.com/content/www/us/en/support/products/96551/graphics/intel-hd-graphics-family/intel-hd-graphics-620.html

When you scroll down, there's two files available to download. I tried to install both of them but neither of them worked. I get the following message from the installer from Intel: "Your system has a driver that is locked to manufacturer specifications. Please try downloading the latest driver from the manufacturer's website to update."
Am I downloading the wrong driver?

I've also disabled media.wmf.zero-copy-nv12-textures. Will let you know how I go.

Thanks for your help!

Flags: needinfo?(eheo4trlh)

Based on this page, there are two ways to download the driver. Does this Inter Driver Assistant work for you? Thanks!

Flags: needinfo?(eheo4trlh)

thanks. I just installed the Intel Driver Assistant and had it scan my computer (funnily enough, it didn't work on Firefox so I had to use Chrome). I then saw the following message:

A customized computer manufacturer driver is installed on your computer. The Intel Driver & Support Assistant is not able to update the driver. Installing a generic Intel driver instead of the customized computer manufacturer driver may cause technical issues. Contact Lenovo for the latest driver for your computer.

earlier I was trying to download the driver from Intel directly and that's why it wouldn't install. so I went to my laptop manufacturer's website to locate the appropriate driver here: https://pcsupport.lenovo.com/us/en/products/laptops-and-netbooks/thinkpad-t-series-laptops/thinkpad-t470p/downloads/ds120580

anyway, it has been successfully updated now

between my previous reply and now, I've had no issues with the video after disabling media.wmf.zero-copy-nv12-textures. should I re-enable it and see if updating the driver has done the trick? or leave it as is?

thanks for your help!

Flags: needinfo?(eheo4trlh) → needinfo?(alwu)

Let's try to re-enable media.wmf.zero-copy-nv12-textures. If you still see the issue, please help us capture a new about:support and the profiled result again. Thanks!

Flags: needinfo?(alwu)
Attached file about:support
so I've re-enabled `media.wmf.zero-copy-nv12-textures` after updating the driver and the videos have started to break again.

here's the profile: https://share.firefox.dev/43C3LQA

these modules were added to the Custom preset: `timestamp,HTMLMediaElement:4,PlatformDecoderModule:5,MediaDecoder:5,HTMLMediaElementEvents:4,AudioSink:5,MediaSource:5,MediaDecoderStateMachine:5,MediaFormatReader:5,AudioSinkWrapper:5`

note that I have to go back to add these modules to `about:logging` each time. I don't see a way to save it

also attached `about:support` to the comment

thanks!
Flags: needinfo?(alwu)

Okay, then could you disable that pref for now, and then we can check whether disabling that pref really helps the issue? In the meanwhile, I will NI my colleague who knows this graphic part more than me to see if he has any idea. Thank you!


Sotaro, zero copy seems broken on the reporter's Intel HD 620 graphic card, do you have any idea about that? Thank you!

Flags: needinfo?(alwu) → needinfo?(sotaro.ikeda.g)

(In reply to housebottle from comment #25)

Here's the profiler data: https://share.firefox.dev/42H0RK9. I could see the video loading in the progress bar but it didn't play.

From the profiler data, there were MediaPDecoder threads in GPU process that were blocked at WMFVideoMFTManager::Shutdown()

mozilla::WMFVideoMFTManager::Shutdown()
RefPtr<mozilla::MFTDecoder>::operator=(void*)
RefPtr<mozilla::MFTDecoder>::assign_assuming_AddRef(mozilla::MFTDecoder*)
RefPtr<mozilla::MFTDecoder>::ConstRemovingRefPtrTraits<mozilla::MFTDecoder>::Release(mozilla::MFTDecoder*)
mozilla::RefPtrTraits<mozilla::MFTDecoder>::Release(mozilla::MFTDecoder*)
mozilla::MFTDecoder::Release()
mozilla::MFTDecoder::~MFTDecoder()
RefPtr<IMFTransform>::~RefPtr()
RefPtr<IMFTransform>::ConstRemovingRefPtrTraits<IMFTransform>::Release(IMFTransform*)
mozilla::RefPtrTraits<IMFTransform>::Release(IMFTransform*)
0x7ff9a30938ca
0x7ff9a30873f4
0x7ff9a3087558
0x7ff9a3096870
RtlEnterCriticalSection
RtlpEnterCriticalSectionContended
RtlpWaitOnCriticalSection
RtlpWaitOnAddress
RtlpWaitOnAddressWithTimeout
ZwWaitForAlertByThreadId

From comment 34, I wonder if videoMFTDecoder is serially destroyed, the problem could be prevented.

Flags: needinfo?(sotaro.ikeda.g)
Assignee: nobody → sotaro.ikeda.g

If D179907 does not address the problem, we might need to block FEATURE_HW_DECODED_VIDEO_ZERO_COPY with Intel KabyLake.

(In reply to Sotaro Ikeda [:sotaro] from comment #39)

hi Sotaro,

thanks for looking into this.

I've downloaded in the zip file in the first link and launched the Nightly browser. used it to watch a couple of videos before the videos broke again. now the playback has stopped working.
I'm not sure what I am meant to do with the second link...

do you need me to run the profiler on the Nightly build?

thanks

Flags: needinfo?(eheo4trlh) → needinfo?(sotaro.ikeda.g)

(In reply to housebottle from comment #40)

I've downloaded in the zip file in the first link and launched the Nightly browser. used it to watch a couple of videos before the videos broke again. now the playback has stopped working.

Thank you for testing the build. It means that D179907 does not address the problem. Then it seems that we need to block FEATURE_HW_DECODED_VIDEO_ZERO_COPY with Intel KabyLake.

Can you also check the following build of D179908? Can you check if FEATURE_HW_DECODED_VIDEO_ZERO_COPY is blocked from about:support? And can you check if the problem is addressed_

https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/L9nE6qJ7RdSXQ1qzViawjQ/runs/0/artifacts/public%2Fbuild%2Ftarget.zip
https://treeherder.mozilla.org/jobs?repo=try&revision=da4d295bba80a727225ea5c4336bd39bd1af9634

do you need me to run the profiler on the Nightly build?

Thank you. It is not necessary for now.

thanks

Flags: needinfo?(sotaro.ikeda.g) → needinfo?(eheo4trlh)
Attachment #9337396 - Attachment is obsolete: true
Attached file about:support
(In reply to Sotaro Ikeda [:sotaro] from comment #41)

hi Sotaro,

I've just checked and HW_DECODED_VIDEO_ZERO_COPY is blocked in `about:support`. however, since I don't know how to intentionally break the playback, I'm just going to have to use this particular build and wait and see if the playback stops at any point. no other way to replicate the bug and verify if the problem has been fixed

thanks again.

Thank you for checking the patch! Then we confirmed that D179908 could block HW_DECODED_VIDEO_ZERO_COPY with Intel HD Graphics 620. You do not need to continue to use the build of D179908. Since same thing could be done by pref media.wmf.zero-copy-nv12-textures = false.

From Comment 30, "media.wmf.zero-copy-nv12-textures = false" seemed to address the problem. And D179907 did not address the problem. I don't have any other ideas to work around the problem other than blocking HW_DECODED_VIDEO_ZERO_COPY.

It seems better to block HW_DECODED_VIDEO_ZERO_COPY with Intel KabyLake.

ah okay. that's great. thank you for your help!
will keep an eye out for when the patch makes its way to a stable release

Flags: needinfo?(eheo4trlh)
Pushed by sikeda.birchill@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/535c66924d75
Block FEATURE_HW_DECODED_VIDEO_ZERO_COPY with Intel KabyLake r=gfx-reviewers,aosmond
Status: NEW → RESOLVED
Closed: 11 months ago
Resolution: --- → FIXED
Target Milestone: --- → 116 Branch
QA Whiteboard: [qa-116b-p2]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: