Open Bug 1692881 Opened 2 months ago Updated 29 days ago

Firefox Nightly seems to not be able to play newer and more recent UHD videos on YouTube

Categories

(Core :: Audio/Video: Playback, defect, P3)

Firefox 87
defect

Tracking

()

UNCONFIRMED

People

(Reporter: kpopempire, Assigned: alwu)

References

(Blocks 1 open bug)

Details

(Whiteboard: [media-youtube])

Attachments

(4 files)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:87.0) Gecko/20100101 Firefox/87.0

Steps to reproduce:

Go to YouTube.com
Play UHD 4K Video 1: https://www.youtube.com/watch?v=f0NdOE5GTgo

Play UHD 4K60FPS Video 2: https://www.youtube.com/watch?v=h3fUgOKFMNU

Play Newer UHD 4K60FPS Video 3: https://www.youtube.com/watch?v=ENnWAqufvCI

Actual results:

UHD 4K Video 1 plays fine.
UHD 4K60FPS Video 2 plays fine.
Newer UHD 4K60FPS Video 3 does not play correctly with over half of the frames dropping.

Expected results:

Video 3 should play correctly the same as Video 1 and 2.

Used YouTube stats for Nerd to record framerate.

Video 3 Freezes multiple times making the video unwatchable unless the video is downgraded to a lesser resolution.

The screenshot shows very fast network speed but over half the frames are dropping.

I have noticed this with other videos as well. It is not just this video.

I have noticed that video 2 is actually newer than video 3 and video 2 plays fine. So the problem is not the newness of the video but something else. They both seem to use the same codecs which makes this seem strange.

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Component: Untriaged → Audio/Video: Playback
Product: Firefox → Core

I can reproduce this.

Assignee: nobody → alwu
Blocks: 4k-video
Severity: -- → S2
Priority: -- → P3
Whiteboard: [media-youtube]

Here is the profiled result where video sink kept dropping frames.

Thank God! I mean I am sorry for this problem, but I was worried it just might only be my system OS.

There are many other videos too not just this one.

I think most of the videos I saw like this seem to be recorded using the same type of device. I could be wrong, but most of them seem to be Kpop fancams which may be recorded using high-end 4K cell phones like the Samsung Galaxy S20 or late-model iPhones.

There is also the possibility that they could be using high-end cameras like RED.

The sideways full-screen recording may be a factor. I have noticed that this type of recording is increasingly becoming more and more popular.

Do you think this could be related to bug 1688947 ? - https://bugzilla.mozilla.org/show_bug.cgi?id=1688947

Attached file Log

There are two problems,

(1) can we make decoding faster?
Bug 1688947 will probably help if we can utilize GPU as much as we can so that the speed of video decoding can catch up with the speed of playing audio. I noticed that watching same 4K video on Chrome would still cause a lot of dropped frames, but video can be played way more smoothly on Chrome.

Therefore, I think the most critical problem is next one.

(2) the mechanism of synchronizing audio and video
Those dropped video frames happened in VideoSink where we do the sychronization for audio and video. However, at the time we presented them, they were late than current audio clock so that we dropped them.

By checking the log, you can see those time differences were accumulated from 12812 us to 940583us, we finally start another mechanism to skip to the next key frame until the time difference was large enough. However, within those period, we had already made video frozen for several seconds. (I think the improvement here should be doing that earlier) Even if we got new frames, we would also not display them immediately, because those video frames were beyond the current audio clock, so the frozen period was actully really long.

Improving this mechanism should make the result way better, and I need to think about what the best way is to do so.

I found another video with excessive frame drops:
https://www.youtube.com/watch?v=v-MW34KSOSM

The second video (see screenshot) is from an entirely different channel and most likely filmed with a different device.

I am getting about 5 to 10% frame drops on Linus Tech Tips 4K videos now. I have the new experimental WebAPI: WebGPU enabled. https://www.youtube.com/watch?v=eS4bNKLxEL0

Per comment14, as this issue is related with the mechanism of synchronizing audio and video, which is not easy to be modified. We might need a full evaluation about that design before doing any modification, which won't happen soon. Sorry about that.

In addition, as this issue won't happen on all 4K videos and the workaround for users exists, which is to watch video with lower resolution. So I'm going to change the severity of this bug to S3.

(In reply to Jeffrey G from comment #19)

I am getting about 5 to 10% frame drops on Linus Tech Tips 4K videos now. I have the new experimental WebAPI: WebGPU enabled. https://www.youtube.com/watch?v=eS4bNKLxEL0

Does enaling that help with other videos which have huge dropped frames?
Thank you.

Severity: S2 → S3

I can't answer the above question because I have reinstalled my system with the new Centos Stream 8 Linux. It could be that because of the good work being done here on this bug I can now play the problem videos stated earlier in this bug report however, I now have a new problem: I am getting a 5% frame drop on ANY YouTube video I watch - even low definition videos:

So, I submitted a new bug: 1695401
https://bugzilla.mozilla.org/show_bug.cgi?id=1695401

I don't know if the new bug I submitted is related to the work being done here.

50% frame drop to 5% frame drop is a huge improvement by the way. Thank you!

Duplicate of this bug: 1699383
You need to log in before you can comment on or make changes to this bug.