1922848 - Certain Videos will not playback after having ran for a while - all h264 decoding halts

Status: UNCONFIRMED

(Core :: Audio/Video: Playback, defect)

Description

[Kinsi]

Attachment: Firefox 2024-10-04 20.57 profile.json.gz

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:131.0) Gecko/20100101 Firefox/131.0

Steps to reproduce:

Watching a Twitch Stream for a while (30-60 Minutes). The attached profile is me refreshing a previously watched mp4 file of when the Bug triggered.

Actual results:

At some point the Video playback will freeze - Audio keeps playing. Trying to then reload the Site will result in the Video loading indefinitely / never playing.

Trying to play back a simple h264 mp4 file at that point will result in the same with it being buffered, but still "loading" indefinitely / never playing

This started happened after the most recent update, everything else equal. Restarting Firefox fixes it immediately until it happens again.

Disabling Hardware Acceleration seems to resolve it.

Expected results:

Nothing out of the ordinary.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Karl Tomlinson (:karlt back Feb 10)]

Isolated Web Content 2 has

• "MTG::MediaTrackGraphStableStateRunnable ControlMessage"s occurring for the entire period.
• 5 x MDSM::Seek markers with associated MDSM::StateChange - COMPLETED. MDSM::PlayStateChanged - PLAY_STATE_ENDED occurs before each seek. These likely correspond to the reloads.

The Remote Data Decoder process has a VPXDecoder decoding frames for the first 1.5s.
Isolated Web Content 1 appears to be the trigger for RDD as it has "PlayVideo" markers for 1.5s and then MDSM::EnterVideoSuspend, but VideoSink::RenderVideoFrames markers repeat for the entire period.

If this occurs again, logging can be added to a profile, which would have more info:

1. Load about:logging
2. Logging preset: "Media playback"
3. Ensure "Logging to the Firefox Profiler" is checked.
4. "Start Logging"
5. Reload the mp4.
6. "Stop Logging". New tab is opened after a few seconds, showing Profiler.
7. "Upload Local Profile".
8. Include everything, but most importantly "hidden threads".
9. Upload.
10. Paste URL here.
11. If you are able to identify in about:processes the process ID corresponding to the tab with the mp4, then that would help in finding it in the profile, if there are multiple tabs open.

Comment 3

[Kinsi]

The seeking was me manually clicking around the timeline on the MP4 to possibly add some more info of interest - I have also done that in this new profile.

Additionally, when I reproduced the issue I have also played back a YouTube Video where this Issue does not show up

2100 is the PID of the MP4 / H264
10012 is the PID for YouTube

https://share.firefox.dev/3NhHcdH

Thank you!

Comment 4

[Karl Tomlinson (:karlt back Feb 10)]

The first video sample is being demuxed, but not decoded.

Web Content PID: 2100
MediaSupervisor #8

{
  "start": 3014423.040115625,
  "end": null,
  "name": "LogMessages",
  "category": 1,
  "threadId": null,
  "data": {
    "type": "Log",
    "module": "PlatformDecoderModule",
    "name": "Sandbox GPU decoder supports requested type video/avc"
  }
}
{
  "start": 3014423.234015625,
  "end": null,
  "name": "LogMessages",
  "category": 1,
  "threadId": 2384,
  "data": {
    "type": "Log",
    "module": "MediaFormatReader",
    "name": "MediaFormatReader[226f7f0de00] ::OnVideoDemuxCompleted: 1 video samples demuxed (sid:0)"
  }
}
{
  "start": 3014423.278915625,
  "end": null,
  "name": "LogMessages",
  "category": 1,
  "threadId": 2384,
  "data": {
    "type": "Log",
    "module": "MediaFormatReader",
    "name": "MediaFormatReader[226f7f0de00] ::RequestVideoData: RequestVideoData(0), requestNextKeyFrame=0"
  }
}
{
  "start": 3014423.297215625,
  "end": null,
  "name": "LogMessages",
  "category": 1,
  "threadId": 2384,
  "data": {
    "type": "Log",
    "module": "MediaFormatReader",
    "name": "MediaFormatReader[226f7f0de00] ::Update: Update(Video) ni=1 no=1 in:0 out:0 qs=0 decoding:0 flushing:0 desc:uninitialized pending:0 waiting:0 eos:0 ds:0 sid:4294967295 waitcdm:0"
  }
}

GPU Process
MediaSupervisor #26

{
  "start": 3014426.216825,
  "end": null,
  "name": "LogMessages",
  "category": 1,
  "threadId": null,
  "data": {
    "type": "Log",
    "module": "PlatformDecoderModule",
    "name": "WMFDecoderModule::CreateVideoDecoder success for manager with description wmf H264 codec software video decoder - no DXVA, yv12 - DXVA failure: Too many DXVA videos playing"
  }
}

Web Content PID: 2100
MediaDecoderStateMachine

{
  "start": 3019559.541915625,
  "end": null,
  "name": "LogMessages",
  "category": 1,
  "threadId": null,
  "data": {
    "type": "Log",
    "module": "MediaDecoder",
    "name": "MediaDecoderStateMachine[226f96bb100] state=DECODING_FIRSTFRAME Not Enough Data to seek at this stage, queuing seek"
  }
}

Comment 5

[Karl Tomlinson (:karlt back Feb 10)]

Can you see if you can reproduce in a Nightly build, please?
If the problem exists there then than would rule out bug 1922278 as the cause of "Too many DXVA videos playing".

Comment 6

[Kinsi]

I tried to replicate it in a (fresh) Nightly install and I cannot reproduce it there.

Comment 7

[Karl Tomlinson (:karlt back Feb 10)]

Thanks! I think that is good news.

If this is related to bug 1922278, then this will be fixed in 132.0b5, which should be available within the next couple of days, or Firefox 132 release at the end of the month.

Comment 8

[Kinsi]

I have updated to 132 today and unfortunately the Bug is still present

Comment 9

[SpencerHakim]

I believe I'm running into this issue too. Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:132.0) Gecko/20100101 Firefox/132.0, using an Nvidia RTX 4070 Super and 4K@120Hz screen.

In Firefox 131, Twitch streams would eventually start stuttering (video only, audio was always fine), with the Twitch player's Video Stats showing Skipped Frames rapidly increasing every second (by at least 50% of the video's framerate) until pausing and resuming the player or refreshing the page.

In 132 this escalated to the video freezing entirely after a momentary stutter (again, audio is fine), with the Skipped Frames increasing by the video's framerate every second. I can pretty reliably reproduce this in under a minute on new profile by doing the following:

1. Install FrankerFaceZ addon from AMO (not actually required, but makes reproduction a lot faster and more reliable)
2. Open any Twitch stream (though I think higher bitrates/framerates might make reproduction easier?)
3. In the bottom right of the player next to the volume slider, click the Audio Compressor button to enable it (you may need to click it twice to get it to actually enable it)
   • This button is added by the FFZ addon and uses a Web Audio API DynamicsCompressorNode. No idea why this would impact video rendering or make this easier to reproduce, but the addon's source is on Github if that's helpful.
4. Mash the Windows and/or Alt keys, switch back and forth between tabs, switch between apps, etc, until the video stutters and freezes

Importantly, step 4 also reliably triggers brief stuttering on other video players (including YouTube, with YouTube's Stats For Nerds reporting dropped frames), but they almost always recover and resume smooth playback. Only once did I manage to get a Video.js demo to break with the above method during the relatively short time I attempted to reproduce this outside of Twitch, and FFZ doesn't touch any of those sites.

Here is the logging profile I captured using the steps in comment #2, the tab was associated with process 2396 https://share.firefox.dev/4eptAYS

Comment 10

[Karl Tomlinson (:karlt back Feb 10)]

I filed bug 1930001 for comment 9 because that sounds a bit different.

Comment 11

[Kinsi]

FWIW: As expected, setting media.hardware-video-decoding.enabled to false prevents this from happening - Previously I had entirely disabled HW acceleration in the settings but thats ofc very undesirable