1933452 - Video freeze after a few seconds, but audio continues. After some time the video resumes

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P2)

Description

[Julien Wajsberg [:julienw]]

STR:

1. Open https://www.youtube.com/watch?v=QKu5p658bOk
2. Play the video.
3. After a few seconds (sometimes 16, sometimes 19) the video freezes, but the sound continues.
4. After more seconds the video seems to resume.

profile from about:logging => https://share.firefox.dev/3V8SRQg

When skipping the video manually we can make the video resume just after the freeze.

Comment 1

[Julien Wajsberg [:julienw]]

Attachment: about-support.json

Comment 2

[Paul Adenot (:padenot)]

Alastor, really strange profile, can you give it a go with your tool so we understand what's going on? I see two frames of av1 taking about 6s (instead of about 80us) to decode in the RDD when this happens.

Comment 3

[Julien Wajsberg [:julienw]]

https://share.firefox.dev/4eL6uMx <= this one should be at 720p all the time, less distractions.

Comment 4

[Alastor Wu [:alwu][PTO until 12/16]]

This issue is probably Linux only, and the reason of video freezing was because we skipped 6 seconds of video frames from here. The detailed investigation is below.

---

• encountered the stream Id change, and recreated a new video decoder : https://share.firefox.dev/49rHKYV
• the first sample sent to the decoder, raised an error : https://share.firefox.dev/3CKD2Jt
• the error resulted in skipping video frame from 18000000 to 24000000 : https://share.firefox.dev/3Vdxc9M
• therefore, we had 6s gap in our decoded video frames : https://share.firefox.dev/4176Utv
• by checking PMediaDecoder threads in the RDD process, we can see no log for 18000000, which means the error was probably raised before sending input to the decoder : https://share.firefox.dev/3OsoUa7

Now the question,
Q: Why did we get that decode error when sending the first frame? Was the decoder not configured properly or something else?
A: I'm suspecting that it's related with VAPPI, per this we can see all previous images were sw based images, maybe we incorrectly tried to create a hw decoder and failed. From the logging, we can even found Video decode error, disable HW acceleration from here.

Comment 5

[Alastor Wu [:alwu][PTO until 12/16]]

Could you try to disable the pref media.hardware-video-decoding.enabled to see if it helps? Thanks!

Comment 6

[Julien Wajsberg [:julienw]]

Yeah exactly, when I disable it and restart Firefox, then the video doesn't freeze anymore.

Comment 7

[Alastor Wu [:alwu][PTO until 12/16]]

Julien, could you help me enable the pref and capture a profile again by using about:logging with the media playback preset and adding an additional log module ,FFmpegVideo:5? Thanks!

Comment 8

[Julien Wajsberg [:julienw]]

Sure, here it is !
https://share.firefox.dev/3B7dkOH

Comment 9

[Alastor Wu [:alwu][PTO until 12/16]]

Julien, could you help me try if this build works for you? Also could you help me capture a profile again by following the instruction in the comment7, so that I can verify whether the issue is solved properly by checking the logs. Thanks!

Comment 10

[Julien Wajsberg [:julienw]]

Hey Alastor !
Yes this seems to work for me :)
Here is the requested profile => https://share.firefox.dev/4gbQi8j

Comment 11

[Alastor Wu [:alwu][PTO until 12/16]]

Attachment: Bug 1933452 - set 'mFirstFrameTime' to prevent skipping video frames in the case of newly created HW decoder fails on decoding the first frame.

When a video error occurs, we have a mechanism in place to skip to the
next frame if next keyframe exists [1]. However, if the error is caused
by the inability to use HW decoding when recreating a decoder, this
mechanism should not be triggered.

By setting 'mFirstFrameTime' in this case, we ensure that we can fall
back to the SW decoder with the correct sample time, preventing the
skipping of frames.

[1] https://searchfox.org/mozilla-central/rev/48556b26b96a32672caa46b9abb6d56c234cd84f/dom/media/MediaFormatReader.cpp#2594-2599

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/834790678f44
set 'mFirstFrameTime' to prevent skipping video frames in the case of newly created HW decoder fails on decoding the first frame. r=media-playback-reviewers,karlt

Comment 13

[agoloman]

https://hg.mozilla.org/mozilla-central/rev/834790678f44