Open Bug 1652540 Opened 5 years ago Updated 7 months ago

Random stuttering on YouTube videos

Categories

(Core :: Audio/Video: Playback, defect)

Product:
Core
Component:
Audio/Video: Playback
Platform:
All
Windows 10
Type:
defect

Tracking

()

People

(Reporter: sworddragon2, Unassigned)

References

Details

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0

Steps to reproduce:

  1. Create a new profile.
  2. Go to https://www.youtube.com/watch?v=FYCjJHhxONc .
  3. On the video player set the video quality manually to the best quality (1080p).
  4. Watch the video and/or repeat watching on smooth animations (e.g. ~1:56 to ~2:01).

Actual results:

A single stutter appears every very few seconds.

Expected results:

The video should play smoothly.

Additional details:

This ticket is the successor of bug #1614083 as the previous report claimed about microstuttering (multiple short stutters in a row) which got fixed over time. In the current Nightly the performance is now slightly better than the non-Webrender path but something is still broken and it is uncertain if this is a Webrender issue or not. As comparison Google Chrome works nearly perfectly smooth.

CC'ing :jrmuizel since he was most active in the previous ticket and provided the final patch to fix the microstuttering. Also he figured out some negative bias resulting in skipped frames so maybe fixing this can improve or even fix the stuttering.

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Component: Untriaged → Audio/Video: Playback
Product: Firefox → Core
See Also: → 1614083
Status: UNCONFIRMED → NEW
Ever confirmed: true
OS: Unspecified → Windows 10
Hardware: Unspecified → All

@jrmuizel: Since the patch from bug #1652181 brings the Webrender performance similar to non-Webrender I was a bit curious about 2 things:

  • Does this change also cause Webrender to derp out on Twitch like non-Webrender does[1]?
  • Do now videos stutter that did never expose the microstuttering from bug #1614083 ?

The good news is Twitch works still fine with Webrender being enabled. The bad news is that I do indeed notice now stuttering in other videos that did never expose microstuttering before. However, this is probably not really a bad news since a retest on those videos with Firefox 78.0.2 exposed the same (non-micro)stuttering as well. It seems I just did never notice this stuttering before while they apparently always existed (probably even before bug #1614083 where all was fine for me). This might be another prove that the patch from bug #1652181 is a huge improvement and the correct fix for bug #1614083 and that those remaining stutters are casually probably hard to notice (especially for someone who does not know they even exist). But of course they are still a significant problem and need to be fixed in the future (but personally I'm now less stressed about them - but time will show this once Firefox 79 is being released and enough time passed in a productive environment to test this).

[1]In todays test it was even very extreme. It is hard to tell what exactly goes on but it just looked like the whole video shaked drastically. On making a close look it looks like this issue is similar to a tearing issue (but it might be somethign else). Objects in the video were duplicated multiple times in multiple directions (probably from multiple frames). However, this sort of tearing/duplication only appeared to any objects at any given time that were moving on the video - things that never moved were looking fine.

Since this issue is slightly different from the previous one I did now also test if disabling hardware acceleration or requesting H.264 videos from YouTube instead of VP9 has any effect:

  • Disabling hardware acceleration via about:preferences#general causes the single stutters to increase quite a bit in frequency and duration but otherwise there were no differences. The CPU/GPU usage of the system was 5%/18% with enabled hardware acceleraton and 32%/16.5% with disabled hardware acceleration compared to the 0%/0% if Firefox was closed.
  • Setting media.webm.enabled to false causes the codec on the YouTube player statistics to change from vp09 to avc1 but the stuttering is the very same for both.

Could you try capturing a profile using the Media preset with the Firefox Profiler? Since those other issues have been fixed it would be helpful to have a fresh profile.

Here is the profile: https://share.firefox.dev/391LeSx

The profile was being captured on the current Nightly with a new profile (local settings). The profile shows playing the full video from the startpost ( https://www.youtube.com/watch?v=FYCjJHhxONc ) which got prebuffered before the capture (including wating a bit for the GC, etc.) to avoid miscellaneous noises.

Hi,
Are you able to profile the process again? Due to bug1653350, now we have new marker to show the jitter happening in compositor.
Thank you.

Severity: -- → S3
Flags: needinfo?(sworddragon2)

Here is the new profile: https://share.firefox.dev/2WKdvbf

Flags: needinfo?(sworddragon2)

Can you count how many seconds elapse between each stutter, on average? I think I see the problem in the profile but I'd like to get an independent confirmation.

Flags: needinfo?(sworddragon2)

(In reply to sworddragon2 from comment #7)

Here is the new profile: https://share.firefox.dev/2WKdvbf

Was this also from playing https://www.youtube.com/watch?v=FYCjJHhxONc ? Your profile shows a video that has 13301 frames, but the video you linked to only has 9049 frames, as far as I can see.

It looks like the problem here is the fact that your display frame rate is not exactly 2x of the video frame rate, when using the audio clock as the reference clock. The display frame rate is ever-so-slightly slower than that.

I think to fix this, we would need to stop using the audio clock as the reference clock. Instead, we would need to resample audio so that the video frames align with the display. That's certainly worth investigating. For example, this post asserts that one cannot "exclusively use the audio clock for timing" - but it's what we do at the moment.

If the display frame rate were exactly 2x of the video frame rate, we would see the following pattern across the entire video: update/still/update/still/update/still/... . However, since the display frame rate is slightly slower, at some point we've drifted enough that we need to skip a "still" frame, and end up with two consecutive "update" frames. .../update/still/update/update/still/update/still/...
In the profile, this double-update (or "skipped still") happens, on average, every 17.2 seconds. It happens at the following profile timestamps:
14.6, 32.0, 49.0, 66.2, 83.4, 100.0, 117.1, 134.5, 151.4, 169.2, 186.4, 203.6, 220.9, 237.8
You can see it in the marker list when looking at the UpdateCompositedFrame markers: https://share.firefox.dev/2WIiHwp
A still frame has "no change" at the end.

Some numbers:
This video has a frame rate of 29.97 fps, as evidenced by the video frame timestamps in the video file: frame 3 is at 33ms and frame 9044 at 301701ms. This gives (9044 - 3) * 1000 / (301701 - 33) = 29.97 fps.

In your profile, I also see the following timestamps:

frame 3993 plays at system time 2968ms - 21.3ms = 2947ms during composition 9929 and has video timestamp 133166000µs
frame 9049 plays at system time 171662ms - 2.3ms = 171660ms during composition 20031 and has video timestamp 301868000µs
frame 11300 plays at system time 246782ms - 8.8ms = 246773ms during composition 24531 and has video timestamp ???

Let's compute the video fps relative to the system clock:
(11300 - 3993) * 1000 / (246773 - 2947) = 29.968092 fps
Good, it looks like the system clock and the audio clock mostly agree.

Let's compute the display fps relative to the system clock:
(24531 - 9930) * 1000 / (246782 - 2968) = 59.88581 fps

So that's the problem. If the display was refreshing at 59.940 fps, there would be no stuttering. But it is refreshing at 59.886 fps. And since we use the video frame timestamps as authoritative (in reference to the audio clock), that means that we expect a "skipped still" every now and then. How frequently, exactly? Well, let's compare the frame rates: 59.940 fps - 59.886 fps = 0.054 fps = 1 frame per 18.5 seconds.

Aside: On my machine, I also see microstuttering on the video, on average every 16.4 seconds - but it's the reverse problem! My display refreshes at 60 fps, so slightly faster than the video. So instead of "skipped stills" I get "skipped updates", or "double stills".

(In reply to Markus Stange [:mstange] from comment #9)

(In reply to sworddragon2 from comment #7)

Here is the new profile: https://share.firefox.dev/2WKdvbf

Was this also from playing https://www.youtube.com/watch?v=FYCjJHhxONc ? Your profile shows a video that has 13301 frames, but the video you linked to only has 9049 frames, as far as I can see.

Yes, the new profile was also done on https://www.youtube.com/watch?v=FYCjJHhxONc . To be more precise: It was done under the same conditions as comment #5 (with the newest available Nightly). However, the new linked profile is the second try I captured. After stopping the first capture I did wait around 2 minutes to start another capture to avoid new noises from the GC, etc. . Eventually this conflicted somehow (if so, I would not have expected issues in such a case)?

(In reply to Markus Stange [:mstange] from comment #8)

Can you count how many seconds elapse between each stutter, on average? I think I see the problem in the profile but I'd like to get an independent confirmation.

When I repeat multiple times from ~1:56 to ~2:01 I usually see one short stutter per try. Sometimes I see none, not unusually even ~2 times in a row (or the stutters were short enough that I just missed them). But also quite often I see 2-3 short stutters per try.

Flags: needinfo?(sworddragon2)

(In reply to Markus Stange [:mstange] from comment #10)

I think to fix this, we would need to stop using the audio clock as the reference clock. Instead, we would need to resample audio so that the video frames align with the display. That's certainly worth investigating. For example, this post asserts that one cannot "exclusively use the audio clock for timing" - but it's what we do at the moment.

https://source.chromium.org/chromium/chromium/src/+/master:media/base/audio_shifter.h;l=22;drc=eb740f109b981723ce5dbbcee507d0f5e6c8bd1a

(In reply to sworddragon2 from comment #11)

(In reply to Markus Stange [:mstange] from comment #8)

Can you count how many seconds elapse between each stutter, on average? I think I see the problem in the profile but I'd like to get an independent confirmation.

When I repeat multiple times from ~1:56 to ~2:01 I usually see one short stutter per try. Sometimes I see none, not unusually even ~2 times in a row (or the stutters were short enough that I just missed them). But also quite often I see 2-3 short stutters per try.

Could you capture a profile where you do this a number of times? And then say in which tries you noticed stutters.

The issue I identified doesn't really explain 2-3 stutters in a 5 second window. So I wonder if there's more going on.

Flags: needinfo?(sworddragon2)

(In reply to Jeff Muizelaar [:jrmuizel] from comment #12)

(In reply to Markus Stange [:mstange] from comment #10)

I think to fix this, we would need to stop using the audio clock as the reference clock. Instead, we would need to resample audio so that the video frames align with the display. That's certainly worth investigating. For example, this post asserts that one cannot "exclusively use the audio clock for timing" - but it's what we do at the moment.

https://source.chromium.org/chromium/chromium/src/+/master:media/base/audio_shifter.h;l=22;drc=eb740f109b981723ce5dbbcee507d0f5e6c8bd1a

This is unrelated. It's to do drift compensation between different audio clock domain.

(In reply to Markus Stange [:mstange] from comment #13)

(In reply to sworddragon2 from comment #11)

(In reply to Markus Stange [:mstange] from comment #8)

Can you count how many seconds elapse between each stutter, on average? I think I see the problem in the profile but I'd like to get an independent confirmation.

When I repeat multiple times from ~1:56 to ~2:01 I usually see one short stutter per try. Sometimes I see none, not unusually even ~2 times in a row (or the stutters were short enough that I just missed them). But also quite often I see 2-3 short stutters per try.

Could you capture a profile where you do this a number of times? And then say in which tries you noticed stutters.

The issue I identified doesn't really explain 2-3 stutters in a 5 second window. So I wonder if there's more going on.

I tried counting the stutters per rewind and writing them down, however, this turned out to be really difficult since I need to focus on the video to not miss the stutters, rewinding every few seconds with the mouse and writing them down blindly with the other hand. This did not work well and so I did try to do at least a capture of this process but very unfortunately the Profiler did show me an error[1][2].

But in this attempt I noticed that the stutters were quite uncommon, estimated a bit less frequent than the issue comment #10 describes (but I might also have missed some very short stutters). This is quite interesting as in bug #1614083 comment #90 I did see those stutters very frequently while a day later in bug #1614083 comment #92 the frequency was already lower roughly up to the test of yesterday and in my test an hour ago the frequency was very low. However, each test has a new setup (a new fetched Nightly and a new profile) so that could explain differences but it still looks a bit weird to me.

The only shared things across those tests is that I could see stutters in every of them. Eventually if the issue you found gets fixed it will already solve the stutters I see. If not it will probably mitigate them and make it easier to find the remaining issues. But one thing that is still open to me is bug #1614083 comment #91. The frequency is only a bit higher than of the issue you found so I'm wondering if :jrmuizel actually sees the same issue. @:jrmuizel: Did you every figure out what the cause of the negative bias/skipped frame is? Could it eventually be the same issue that comment #10 describes? Or is there already a report where the progress of it can be tracked?

[1]The error was:

Firefox Profiler
Couldn't retrieve the profile from the Firefox.

NotFoundError: IDBDatabase.transaction: 'symbol-tables' is not a known object store name

The full stack has been written to the Web Console.

And the warnings/errors in the web console were:

Successfully compiled asm.js code (total compilation time 73ms) zee-worker.js
Some cookies are misusing the “SameSite“ attribute, so it won’t work as expected 3
Cookie “_ga” has “SameSite” policy set to “Lax” because it is missing a “SameSite” attribute, and “SameSite=Lax” is the default value for this attribute. analytics.js:28:486
Cookie “_gid” has “SameSite” policy set to “Lax” because it is missing a “SameSite” attribute, and “SameSite=Lax” is the default value for this attribute. analytics.js:28:486
Cookie “_gat” has “SameSite” policy set to “Lax” because it is missing a “SameSite” attribute, and “SameSite=Lax” is the default value for this attribute. analytics.js:28:486
Encountered error while cleaning out database: DOMException: IDBDatabase.transaction: 'symbol-tables' is not a known object store name symbol-store-db.js:151:18
onsuccess symbol-store-db.js:151
DOMException: IDBDatabase.transaction: 'symbol-tables' is not a known object store name AppViewRouter.js:80:18
renderCurrentRoute AppViewRouter.js:80
render AppViewRouter.js:116
React 7
Bi
Fi
gs
cl
sl
Qs
Zo
unstable_runWithPriority scheduler.production.min.js:19
React 4
Wo
Zo
$o
el
Redux 6
notify
notifyNestedSubs
handleChangeWrapper
p
zv
dispatch
Su receive-profile.js:918
Uncaught (in promise) DOMException: IDBDatabase.transaction: 'symbol-tables' is not a known object store name 3d27ca430016f9e559da.bundle.js:68

[2]This is not the first time that I did get an error from trying to get a profile as my first attempt on the previous bug at bug #1614083 comment #24 did throw an error too (which is quite a bit unfortunate since setting all up and doing eventually a few captures to get a good profile needs quite some time - usually close to an hour and sometimes even a bit longer).

Flags: needinfo?(sworddragon2) → needinfo?(jmuizelaar)

(In reply to sworddragon2 from comment #15)

but very unfortunately the Profiler did show me an error[1][2].

Oh no, that's terrible! :(
Sorry about all the wasted time. The profiler bug is tracked in bug 1653276 and you can fix it by following the steps from bug 1650627 comment 2.

If you try this again, don't worry about taking super precise notes. Sure, they would be useful, but I don't want to create too much work for you. I'm more looking for statements like "I saw n-m frame stutters in most of them, except at the beginning there were a few tries that went smoothly".

Thanks for your time!

I was able to capture a profile: https://share.firefox.dev/32QURlM

I did 100 tries (99 rewinds) and noticed 23 stutters and additionally 7 stutters where I'm not sure (so 23-30 stutters). But there might be a slight counting inaccuracy on keeping track of this numbers of roughly 5%. While the first and last try are hard to track (due to starting/stopping) the first stutter I noticed was in try 5 or 6 (I think it was actually 6). As a side-note I noticed the first time on those tests at some point (I think a bit before the half of the capture) a loading circle (and a halt) in the YouTube player for around a second (the video got fully prebuffered before the capture).

One thing that looks strange to me is the running time of 227.102 seconds the profile shows. Every try needs 5-6 seconds (due to manually rewinding there might be a slight offset for each try of +/- 0.x seconds) which should result in 500-600 seconds, probably a bit more since the 5% counting inaccuracy likely counts more to additional tries instead of less.

Thanks! These numbers match up with the hypothesis: We expect a single frame of stutter every 18 seconds or so. The sequence you selected is 5-6 seconds long, so we expect a stutter every third or fourth rewind. So, observing a stutter on 23-30 out of 100 tries matches the explanation exactly.

What's more surprising to me is the fact that Chrome is not having these stutters. From what I can tell, the strategy that Firefox uses (dropping or repeating a frame on occasion in order to sync up slightly mismatched frame rates) is the default in almost all audio players, and I'd be surprised if Chrome did things differently.

(In reply to sworddragon2 from comment #17)

One thing that looks strange to me is the running time of 227.102 seconds the profile shows. Every try needs 5-6 seconds (due to manually rewinding there might be a slight offset for each try of +/- 0.x seconds) which should result in 500-600 seconds

This means that the profiler ran out of buffer. You can increase the buffer size in the settings if you want. But your profile contained all the necessary information anyway.

(In reply to Markus Stange [:mstange] from comment #18)

What's more surprising to me is the fact that Chrome is not having these stutters.

Eventually Chrome does not use the audio-clock as the only reference clock similar to your conclusions in comment #10. Do we have any more informations since then if this is the way to go or what else we could do? I'm throwing this in as the ticket seems to stall a bit and might be forgotten for a long time otherwise. Not to mention that we could get on par with Google Chrome's video playback performance.

But I don't want to rush anybody about this - with the last ticket being solved the smoothness is now back to its normal level and I have no urgent need for perfect smoothness and can still wait especially if a fix turns out being quite complex and time-consuming.

Flags: needinfo?(mstange.moz)

I do not have any new information, and I haven't looked into what Chrome is doing. I agree it would be nice to get to the bottom of this, but for now there are more important things to do.

Flags: needinfo?(mstange.moz)

Alastor, I think you may close this old issue as per Bug 1433531#c9.

Flags: needinfo?(alwu)

Due to the recent improvements on VP9 video playback out of curiosity I checked 3 weeks ago how well the video from this report works now. To my surprise it had permanent stuttering and there was barely any second which was free from it - the issue got drastically worse. However, on a more closer look I noticed the video was now deliver via H.264 and not VP9 (as it was in the past) so I can't tell if this stuttering was always the case here and thus more or less normal. I also noticed that other videos that were delivered in the past via VP9 got now changed to H.264 and this was also the case for another user on another browser I asked testing this - I don't know why YouTube went a step back to favoring more H.264 where they have or had VP9 encoded content.

But I can't tell if YouTube would still deliver VP9 variants of affected videos - it is also difficult to test this since H.264 can't be disabled in Firefox for testing purposes.

Edit: To make things a bit more clear: YouTube still delivers some (even older videos) via VP9 but some others just got changed to default to H.264 (including the linked one in this report) for some reason.

I couldn't reproduce this issue on Fx128.0.3 on my Windows11, and I also saw H264 on that video.

Would you mind to capture a profiled result for that? If you are using Fx129+, then simply follow this instruction. If not, after selecting media playback preset, please replace existing log modules by HTMLMediaElement:4,HTMLMediaElementEvents:4,cubeb:5,PlatformDecoderModule:5,AudioSink:5,AudioSinkWrapper:5,MediaDecoderStateMachine:4,MediaDecoder:4,MediaFormatReader:5,GMP:5,EME:5,MediaSource:5,MediaSourceSamples:5.

Thank you!

Flags: needinfo?(sworddragon2)
Flags: needinfo?(jmuizelaar)
Flags: needinfo?(alwu)

Here is the profile: https://share.firefox.dev/4cdjb1k

It was recorded on Firefox 128.0.3 on my normal user profile with the modules you mentioned. Before recording I did set the video quality to 1080p and disabled autoplay and then I just did record the entire video running once. The stuttering was still present all the time but while I consider it pretty bad it might be important to know that those stutters might be still hard/impossible to see for an user if they are not aware of them. Also a few seconds after the video played I noticed the video freezing for around a second which never happened here before (maybe it just happened due to the profiling?).

Flags: needinfo?(sworddragon2)
Duplicate of this bug: 1778586
You need to log in before you can comment on or make changes to this bug.