Recent random microstuttering on YouTube videos
Categories
(Core :: Graphics, defect, P3)
Tracking
()
| Tracking | Status | |
|---|---|---|
| firefox-esr68 | --- | unaffected |
| firefox-esr78 | --- | disabled |
| firefox78 | --- | wontfix |
| firefox79 | --- | fixed |
| firefox80 | --- | fixed |
People
(Reporter: sworddragon2, Assigned: jrmuizel)
References
(Blocks 1 open bug, )
Details
(Keywords: regression)
Attachments
(3 files, 1 obsolete file)
Profile event delay
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0
Steps to reproduce:
- Make sure hardware acceleration is enabled.
- Go to https://www.youtube.com/watch?v=FYCjJHhxONc.
- On the video player set the video quality manually to the best quality.
- Watch the video and/or repeat watching on smooth animations.
Actual results:
Every ~15-30 seconds microstuttering appears that lasts for roughly 0.5-2 seconds.
Expected results:
The videos should play smoothly.
Additional details:
- A Phenom II X6 1045T as CPU and a GeForce GTX 1050 Ti as GPU on Windows 10 version 1909 (64 bit) are being used.
- I don't remember to have seen this issue some months ago and earlier.
- The issue does not appear on all but on my testing on quite a bunch of videos.
- The microstuttering does not always appear on the same location as it is random when it appears.
- The video qality does not matter as the microstuttering does even appear on 144p but setting the quality manually to the highest value makes it easier for consistent testing and to distinguish easier between smooth playing and microstuttering.
- The microstuttering also appears when the network buffer bar on the video is far ahead.
- Testing against a Firefox nightly from around a year ago with default settings still causes this issue.
- Testing against a graphics driver from around a year ago does still cause this issue.
- Disabling hardware acceleration does fix this sort of intervaled microstuttering but causes more constant shorter FPS drops to appear (which subjectively appears to be much smoother).
- Downloading and testing Google Chrome causes the video to run flawlessly out of the box.
I tried to simplifly anyting above since there are many informations involved to avoid too much confusion. Based on these observations my guess was that within the last few months either a Windows Update introduced a bug causing this behavior or that YouTube made changes to its site causing those issues on Firefox. While the first is possible it is probably more unlikely since this means Google Chrome would have integrated intentionally or unintentionally already a workaround for this issue. My guess is that simply changes on YouTube caused Firefox to be being behind now as it might require optimizations for new techniques to be applied, etc. Eventually it might be also just a bug or something similar but the main question is simply what does Google Chrome different here to work flawlessly.
The current workarounds are currently to either disable hardware acceleration for a smoother experience or to use Google Chrome on YouTube for a flawless experience.
|
||
Updated•5 years ago
|
Me:
I spotted this yesterday. I see things like this randomly on videos. If i go back 10 seconds where the stuttering was is smooth now.
If i try to go back 10 seconds, wait for 10 seconds, go back 10 seconds and repeat this, at the some time the stuttering will appear again.
This thread:
I try to disable the Hardware Acceleration to see if will fix the problem. The response is NO.
The stuttering is more easy to see and is how you say. Approximately 10-15 seconds.
I use the https://www.youtube.com/watch?v=1La4QzGeaaQ in my test because is moving all the time.
But strange.
|
Reporter | |
Comment 2•5 years ago
|
||
(In reply to Andronachi Marian from comment #1)
I use the https://www.youtube.com/watch?v=1La4QzGeaaQ in my test because is moving all the time.
I did watch the entire video but was not able to reproduce the microstuttering on it (while I still can reproduce it on the video I posted). I also needed to watch it in 1080p since my network can't handle 4k+ reliable and my current replacement GPU (GeForce GTX 650) seems to have troubles on 4k and above too. Maybe your stuttering is caused by another issue. Is your network/system able to process this video flawlessly? From my experience current systems need usually a VP9 codec for advanced hardware accelerated decoding capabilities (for example from the Microsoft Store) to avoid choppy playback.
Could you navigate to about:support and use the button there to copy that information to the clipboard then past it onto this bug, please?
|
|
Reporter | |
Comment 4•5 years ago
|
||
|
|
Reporter | |
Comment 5•5 years ago
|
||
(As additional note I did update Firefox to version 73.0 but the issue still exists).
From my own check on this information I noticed that gfx.crash-guard.status.wmfvpxvideo is set to 2 which I think means a recovery from a crash happened in the past (which might have an influence on hardware acceleration capabilities being utilized). But since I was able to reproduce this issue with past versions of Firefox with a new profile I would not have assumed much relevance at first but I still gave it a try by resetting gfx.crash-guard.status.wmfvpxvideo, gfx.crash-guard.wmfvpxvideo.appVersion, gfx.crash-guard.wmfvpxvideo.deviceID, gfx.crash-guard.wmfvpxvideo.driverVersion and restarting Firefox. As expected nothing did change as I was still able to reproduce the microstuttering but I noticed also something strange: While all those prefenrences did recreate automatically as expected gfx.crash-guard.status.wmfvpxvideo was set to 2 already while no crash happened meanwhile.
That disabling hardware acceleration helps suggests some unhappiness with the gfx adapter. That about:support info suggests that you have 2 different adapters. It indicates the following:
- NVIDIA GeForce GTX 650
- ATI Radeon 3000 Graphics
Is that correct?
|
|
Reporter | |
Comment 7•4 years ago
|
||
Yes, the ATI Radeon 3000 Graphics is the graphics chip built onto the mainboard while the GeForce GTX 650 (GeForce GTX 1050 Ti shortly before and GeForce GTX 1650 now - and the issue reproduces on all 3 of them) is the dedicated graphics card in the PCIe x16 slot. However, the onboard GPU is never being used and no setup to switch between those both graphics adapters is being used. A few days ago I did also change the setting in the BIOS to initialize the PCIe slot first (instead of the onboard one) which causes the onboard GPU not even being registered in Windows's device manager but the issue still exist.
Does the issue still happen when testing in nightly?
If so, does setting the media.wmf.low-latency.force-disabled pref to true have any impact on the issue?
|
||
Comment 9•4 years ago
|
||
me too. even worse than what you're describing honestly, and that's with a 1080 Ti and threadripper 1950X. it just started like 5 days ago. on Nightly 75, windows 10 1903
|
|
Reporter | |
Comment 10•4 years ago
|
||
On testing with the current nightly and a new profile the issue still reproduces. Additionally setting media.wmf.low-latency.force-disabled to true does not give a different result.
The last days I did also do some tests to try to figure more out about this issue which also included altering several settings that might be relevant (including media.wmf.* settings). Nothing did change the behavior with one exception. But here is the result of what I figured out so far:
- Setting media.webm.enabled to false changes the used codec according to YouTube's video player stats from vp09 to avc1 but the issue still exists.
- The issue also still exists when entering fullscreen.
- Disabling the sound via the tab did also not result in any difference.
- Disabling e10s resulted in a significant improvement by reducing the occurences/length of the microstuttering but it did still not fully fix the issue. However, it might be a better workaround than just disabling hardware acceleration since hardware acceleration with this workaround is still available. (But I'm not sure if this means that this has something to do with e10s as I guess it might be just normal processing overhead from the IPC logic once e10s is enabled).
|
|
Reporter | |
Updated•4 years ago
|
|
|
Reporter | |
Comment 11•4 years ago
|
||
Since I finally figured now out how to disable the crash guard (by setting the environment variable MOZ_DISABLE_CRASH_GUARD to 1) I gave it a try (and also deleted the gfx.crash-guard.* settings and restarted Firefox). However, while the gfx.crash-guard.* settings do not recreate anymore when playing the video the microstuttering does still appear.
I'm now a bit curious why the gfx.crash-guard.* settings do set otherwise and if MOZ_DISABLE_CRASH_GUARD does really reliable disable the crash guard here. But maybe the crash guard has just nothing to do with the microstuttering.
It sounds like the issue persists
- irrespective of if crash guards are enabled
- irrespective of if youtube is playing avc1 or vp09
which has me wonder if the issue could be happening in our graphics pipeline (following the decoding of the media).
If you right click on the youtube video and select the stats for nerds option, do the stats report dropped frames in line with the stuttering?
Since this could be graphics related it would be interesting to know if toggling any of the following prefs does anything:
gfx.direct3d11.use-double-bufferinggfx.webrender.force-disabled
Regarding the crash guards, the code here will disable crash guards based on setting of env vars. If you're running a nightly build there is an additional check for non-wmf-vpx guards, but that shouldn't apply to this case. My understanding is that if you disable the crash guards, the prefs will remain set, but the crash guards shouldn't block WMF VPX decoding. If you do not set the env var, then provided the crash guard prefs match your Firefox version, gfx card, and gfx drivers, WMF will not be used to decode vpx.
|
|
||
Comment 13•4 years ago
|
||
actually, i'm getting much worse stuttering when other non-firefox processes are performing cpu-limited operations at like 50% usage or above. in normal use the stutters are very rare. they also happen way, way more when the tab with the active youtube player is not selected. the stutter has a very pronounced effect on the audio latency, almost painful to my ears if wearing headphones, but the effect on the video is relatively subtle. it also causes the mouse and any scrolling to stutter. all of this is basically synchronized, so when a stutter happens this all happens at once. i have tested on a new, blank profile, no difference. i have tested with webrender off, with low latency force disabled, with double buffering on and off, with vp9 disabled. no difference at all. the only thing i've seen that affects it is what i'm doing in the background. maybe this is not the same problem everyone else is discussing since it doesn't seem graphics related at all on my end. it can still stutter when gpu usage is at 2%, and if i run a gpu benchmark while playing a youtube video in the background, it still won't stutter if cpu usage is low.
i initially noticed this happening when running long automation tasks like bulk compression/archival. i was curious as to the cause since it seems to be triggered by non-firefox processes. obviously if the cpu is under a big synthetic load, any program will stutter, but these kinds of operations never caused stuttering before, in firefox or in any other application. i don't get any stutters like this in adobe after effects/premiere or in 3ds max. just in firefox 75. i tried a regression to 74 and didn't notice any stuttering, but of course my real installation might be more prone to this issue owing to the giant profile folder and a session store with like 200 tabs.
anyway i tried playing a youtube video in the background while running a heavily modded installation of skyrim, which should make gpu usage fluctuate between 70 and 90% and cpu usage average 20% but peak at above 90% at times. i didn't notice a single stutter during this. i'm not seeing this issue on my other desktop which has a 9920X, or my laptop which has a 9980HK. maybe the particular issue i'm having is unique to AMD chips. i've tried updating windows and chipset drivers by the way.
|
|
Reporter | |
Comment 14•4 years ago
|
||
(In reply to Bryce Seager van Dyk (:bryce) from comment #12)
If you right click on the youtube video and select the stats for nerds option, do the stats report dropped frames in line with the stuttering?
I get a constant 5% FPS drop (a very suspicious too exact 5% as it never gets even to something like 4.5% or 5.5%) which does not get a burst once I see the microstuttering. However, 5% seems to be pretty high here but I'm unsure if this is actually more or less normal.
(In reply to Bryce Seager van Dyk (:bryce) from comment #12)
Since this could be graphics related it would be interesting to know if toggling any of the following prefs does anything:
gfx.direct3d11.use-double-bufferinggfx.webrender.force-disabled
- On setting only gfx.direct3d11.use-double-buffering to true the issue does still exist.
- On setting only gfx.webrender.force-disabled to true the occurences and duration of the microstuttering do significantly decrease (estimated to 10%-20% of the issue). It is good enough that this issue would be probably now casually hard to notice (and it seems for now to be an acceptable workaround for me). But the microstuttering does still occur in random manners on a still too high frequency that I would consider this not to be a fix - something is still wrong here.
|
|
Reporter | |
Comment 15•4 years ago
|
||
I did do some additional tests:
- On checking again if Google Chrome really works that flawlessly: Very rarely there is a single stutter for a very short moment. It appears to be so insignificant that I would even consider this as being normal (eventually caused by disk-I/O or some other background task (Windows maintenance, etc.).
- On checking the FPS drops on Google Chrome: After playing the full video from the first post the statistics list 2/9058 dropped frames (0.02%) which is much less than the 5% from Firefox.
- On checking the total CPU/GPU usage of the system on maximum quality after the video was fully buffered (including overhead from the task manager):
- Google Chrome: 3% CPU usage and 10% GPU usage (the GPU usage does come only from a Chrome process).
- Firefox without Webrender: 3%-4% CPU usage and 17%-18% GPU usage (around less than 1% CPU usage and 6.5% GPU usage do come from the desktop manager).
- Firefox with Webrender: 4%-5% CPU usage and 17%-18% GPU usage (around less than 1% CPU usage and 5.5% GPU usage do come from the desktop manager and around less than 1% CPU usage and 2% GPU usage do come from the System process).
So Google Chrome does have less FPS drops and less CPU and significant less GPU usage. I guess the extra CPU usage for Firefox does come from the overhead from utilizing more system processes but I'm curious why the GPU usage gets so much higher and if this has actually to do something with this issue. The FPS drops probably do at least affect this issue slightly to the negative. @:bryce If you let the full video from the first post play on maximum quality how much FPS drops do you get? I'm a bit curious if it is 5% on your system too.
|
|
Reporter | |
Comment 16•4 years ago
|
||
Edit: I just noticed the term "FPS drops" in my last 2 posts is not accurate and might be misleading - I actually meant frame drops.
My stat block watching the video from comment 0 through (didn't maximize, but left tab in foreground).
Video ID / sCPN
FYCjJHhxONc / 5V72 RNXY C162
Viewport / Frames
1280x720*1.50 / 0 dropped of 9058
Current / Optimal Res
1920x1080@30 / 1920x1080@30
Volume / Normalized
100% / 100%
Codecs
vp09.00.51.08.01.01.01.01 (248) / opus (251)
On setting only gfx.direct3d11.use-double-buffering to true the issue does still exist.
Just to be sure: this was set to false previously? So the value of this has no impact on the issue.
I'm going to move this to gfx because:
- The issue persists when using software or hardware vpx decoders + when using h264 instead of vpx. This suggests the problem is not codec specific.
- Turning off webrender causes an improvement.
Please move this back to media if new information suggests a media issue, but I think gfx would be better suited to further debug the issue at this stage.
|
|
Reporter | |
Comment 18•4 years ago
|
||
Thanks for testing against the frame drops (although it has been done only at 720p I also get a straight 5% frame drop in 720p). Since you do not get any frame drops at all I did investigate a bit more and figured out that setting privacy.resistFingerprinting to true is responsible for it. This is probably intended and I did create bug #1617380 for it to handle/improve this.
(In reply to Bryce Seager van Dyk (:bryce) from comment #17)
On setting only gfx.direct3d11.use-double-buffering to true the issue does still exist.
Just to be sure: this was set to false previously? So the value of this has no impact on the issue.
Yes, it was set to false (the default value) before.
|
||
Comment 19•4 years ago
|
||
So if you set privacy.resistFingerprinting to false you don't get dropped frames ?
I already have privacy.resistFingerprinting = false, no dropped frames when the shuttering is happening and still 0% dropped frames on youtube.
Still the same with Hardware Acceleration, no dopped frames when the shuttering is happeing.
|
|
Reporter | |
Comment 20•4 years ago
|
||
(In reply to Andronachi Marian from comment #19)
So if you set privacy.resistFingerprinting to false you don't get dropped frames ?
This is correct (but the microstuttering still appears).
|
|
Reporter | |
Comment 21•4 years ago
|
||
Since I have now gfx.webrender.force-disabled set to true for a few days I noticed some strange things:
- While Twitch did run flawlessly so far with Webrender being enabled I noticed some microstuttering and some big short stutters there. Initially I though it might be just the Streamer having a bad connection but enabling Webrender again fixed those issues immediately.
- Minimizing Firefox causes a minor graphic glitch by drawing a few pixel big grey line on the very top of the entire title bar for a blink of an eye. Enabling Webrender again did also fix this issue. I figured also out that this graphic glitch happens with Webrender enabled when gfx.webrender.force-angle is set to false.
I guess this 2 issues are not being too unexpected since non-Webrender is not the default and I would guess it has then just a lower performance by nature. But this also means disabling Webrender to mitigate the YouTube issue is now not a viable workaround anymore since it breaks other things that worked flawlessly before.
(In reply to sworddragon2 from comment #21)
Since I have now gfx.webrender.force-disabled set to true for a few days I noticed some strange things:
- While Twitch did run flawlessly so far with Webrender being enabled I noticed some microstuttering and some big short stutters there. Initially I though it might be just the Streamer having a bad connection but enabling Webrender again fixed those issues immediately.
Should that first enabled be a disabled?
|
|
Reporter | |
Comment 23•4 years ago
|
||
(In reply to Bryce Seager van Dyk (:bryce) from comment #22)
Should that first enabled be a disabled?
No - to clarify this a bit: In the past few months with Webrender enabled (the default setup):
- YouTube has the microstuttering this ticket describes (but only on some videos).
- I never noticed any issues on Twitch (except the obvious ones for example if the Streamer has frame drops).
With Webrender being disabled (gfx.webrender.force-disabled set to true):
- The issue on the affected YouTube videos did heavily reduce (but not fully fix).
- Twitch has microstuttering and normal stuttering that cause now a bad experience.
- The graphics glitch on the title bar on minimizing happens (probably due to disabled ANGLE).
|
|
Reporter | |
Comment 24•4 years ago
|
||
The last days I did do some more tests to try to figure something out but so far I had no success.
- Also I did remove the Microsoft Visual C++ 2015-2019 Redistributable (x64 and x86) to make sure Firefox can only use its own shipped version but the issue did still exist.
- I also tried to profile the issue - however, even with programming knowledge this is not easy to do if you are not trusted with the tools, etc. and have only a vague idea what you are looking for:
- On making a look as good as I could I couldn't see something suspicious with the runtime inspector. Garbage collection seems to be not the cause here when the microstutter strikes and analyzing the FPS was not reliable (as I got rare random big FPS drops; interacting with the website to replay parts obfuscated the result; It is hard to keep track of the FPS counter and the video for microstutter at the same time).
- I also tried the Gecko Profiler addon but it seems to have some issues. Apparently all profiling results have to be uploaded to profiler.firefox.com which is not that optimal - however, for debugging this issue it should be good enough. But also https://profiler.firefox.com always shows the error message "InvalidStateError: A mutation operation was attempted on a database that did not allow mutations." when I try to capture a result so it seems to be currently not working.
For now I'm out of ideas what to do. I could not find what is actually causing this regression. The only 2 things I did not test against as they are tricky or even impossible to do are testing an older Windows 10 state and testing an older YouTube state. YouTube does still have an option to disable its Polymer design but this did also not fix the issue - but it is hard to tell how accurately it reverts the website (for example it might be just the visuals and less the backend).
If somebody has still an idea what to do now feel free to post it - as I would like to avoid using Google Chrome or something similar as a main browser for web medias on the internet.
|
|
Reporter | |
Comment 25•4 years ago
|
||
Out of curiosity (and since this issue annoys me too much/I have currently a moderately amount of free time) I did a mozregression just to see if this issue was at any time in the past better. This was not easy to do and I had to retry the bisecting process a few times as I encountered overall some minor stutters (similar to Google Chrome but with a bit higher frequency which is also always different over the course of time via different builds) and thus the main goal became to find the last known good build/first know bad build where the big chunks of microstuttering strike.
I was successul getting a result but due to the difficulties above the result might be a bit unstable but I did a double check to make sure it is as accurate as possible:
Last known good build: mozilla-central 2018-09-12
First known bad build: mozilla-central 2018-09-13
The good build has some minor stutters which rarely look like a bigger strike of microstuttering but I have to watch quite for a while and then still am never sure (it might be unlucky cases where 2 or 3 minor stutters are randomly striking in a streak). But on the bad build I get longer microstuttering quite often (around every 30 seconds) which makes the issue quite obvious there.
After this result mozregression-gui seems to try to find a closer range but always fails with the error message "Unable to find enough data to bisect.". On looking around I would assume this is due to bug #1610427.
|
||
Comment 26•4 years ago
|
||
The priority flag is not set for this bug.
:jbonisteel, could you have a look please?
For more information, please visit auto_nag documentation.
|
||
Comment 27•4 years ago
|
||
No - to clarify this a bit: In the past few months with Webrender enabled (the default setup):
YouTube has the microstuttering this ticket describes (but only on some videos).
You can you share any links to the videos where the stuttering is worse with WebRender?
|
|
||
Comment 28•4 years ago
|
||
Is possible to try to check this if is happening on other system like my with the Hardware Acceleration: Disabled ?
The comment: https://bugzilla.mozilla.org/show_bug.cgi?id=1614083#c
I have: Intel HD 4600, 8GB RAM, GA-H87-HD3, I5-4440 3.1GHz, Windows 10 1909 18363.657
Maybe is the same problem.
|
|
Reporter | |
Comment 29•4 years ago
|
||
(In reply to Jessie [:jbonisteel] pls NI from comment #27)
You can you share any links to the videos where the stuttering is worse with WebRender?
The video in the URL field/the startpost has the microstuttering. You can just follow the STR from bug 1614083 comment 0 to check it. Personally mostly I just replay it from ~1:56 to ~2:01 as the microstuttering is easy to see in this animation (usually I need here around 2-6 replays to get the microstuttering once).
(In reply to Andronachi Marian from comment #28)
Is possible to try to check this if is happening on other system like my with the Hardware Acceleration: Disabled ?
The comment: https://bugzilla.mozilla.org/show_bug.cgi?id=1614083#c
I think your comment got a bit messed up/is slightly confusing.
|
|
Reporter | |
Updated•4 years ago
|
|
|
||
Comment 30•4 years ago
|
||
Rares - can you see if you can try and reproduce the worse stuttering in videos mentioned in https://bugzilla.mozilla.org/show_bug.cgi?id=1614083#c29 ? https://bugzilla.mozilla.org/show_bug.cgi?id=1614083#c23 seems to suggest this stuttering is worse with WebRender and would like to see if we can repro it. Thanks!
|
||
Comment 31•4 years ago
|
||
Hi I was able to reproduce this issue in our latest Nightly build as well as Beta and Release on a Laptop with Intel UHD 630 and a second Nvidia Geforce GTX 1050. Please note that its barely noticeable on my end and sometimes it just slows down the frames for like half a second, in older builds however it was a lot worse it will just freeze for a few frames and then continue so I tried to get a regression range on this issue.
I'm not sure how accurate it is, I was mostly marking them Good or BAD when it was more noticeable but I can't trully tell if the GOOD builds were actually good or I just missed the frame lag.
Either way i hope these help in some way :
|
|
||
Comment 32•4 years ago
|
||
Glenn, there are a bunch of things going on in this bug but I am mostly interested first in the potential for more stuttering on some videos with WebRender. Not sure if the regression range is accurate - thoughts?
|
|
||
Updated•4 years ago
|
|
||
Comment 33•4 years ago
|
||
The Peru video is very good for noticing the stutters:
https://www.youtube.com/watch?v=1La4QzGeaaQ
Comparing builds from:
- 2019-06-12: Very minimal stutter
- 2019-06-14: Very minimal stutter
- 2020-03-20: Noticeable stutter and even noticed a ~0.5 second hang
Chrome is buttery (french butter) smooth on this video.
I'll see if I can get a profile.
|
|
||
Comment 34•4 years ago
|
||
|
|
||
Comment 35•4 years ago
|
||
Managed to capture a bigger stutter:
https://bit.ly/2wsc7QS
Look around 144.5s when the stutter occured. I looks like the webcontent thread has a 277ms event processing delay, I think caused by the DOM Worker thread.
@jrmuizel: Could you take a quick look at the profile?
|
|
||
Comment 36•4 years ago
|
||
Another profile here, this time showing the microstutters, which looks like a different issue than the event delay above.
https://bit.ly/3bsMUEr
with microstutters around:
~48s
~63s
~118s
~147s
~156.4s
Method: Started profiler at same time as Stopwatch and recorded time when I saw a visible micro stutter.
|
Assignee | |
Updated•4 years ago
|
|
|
Reporter | |
Comment 37•4 years ago
|
||
I did test this issue against the current version of the Nightly to see if the fix from bug #1628137 does change anything.
With gfx.vsync.use-waitforvblank set to false: The microstuttering does still appear. It was a short test but it looked like the frequency in which the microstuttering appeared did very drastically increase compared to past affected versions of Firefox - but I might have been just unlucky here.
With gfx.vsync.use-waitforvblank set to true: The microstuttering does still appear too, however, it looked like the frequency, intensity and length was much less than in past affected versions of Firefox. But it also was a short test and I might have been just lucky here - but if not the microstuttering is still too high to say that all is fine now.
|
|
Assignee | |
Comment 38•4 years ago
|
||
Kris, can you see if you can still reproduce this issue with gfx.vsync.use-waitforvblank set to true. If you can you get a GPUview capture?
|
|
||
Updated•4 years ago
|
|
|
||
Comment 39•4 years ago
|
||
@jrmuizel:
With waitforvblank, the videos feels a whole lot less stuttery. I compared on/off and against chrome and to me it feels like this brings it on par to chrome.
I'll do some more testing tomorrow to make sure I'm not imagining things.
|
||
Updated•4 years ago
|
|
|
Assignee | |
Updated•4 years ago
|
|
|
||
Updated•4 years ago
|
|
|
||
Comment 40•4 years ago
|
||
Stuttering is definitely still happening and quite obvious.
On the Renderer thread it looks like most time is spent in:
virtual long CDevice::GetDeviceRemovedReason from d3d11.dll
called by
mozilla::wr::RenderCompositorANGLE::WaitForPreviousGraphicsCommandsFinishedQuery(bool)
mozilla::wr::RendererOGL::WaitForGPU()
mozilla::wr::RenderThread::UpdateAndRender(mozilla::wr::WrWindowId, mozilla::layers::BaseTransactionId<mozilla::VsyncIdType> const&, mozilla::TimeStamp const&, bool, mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits> > const&, mozilla::Maybe<mozilla::wr::ImageFormat> const&, mozilla::Maybe<mozilla::Range<unsigned char> > const&)
mozilla::wr::RenderThread::HandleFrameOneDoc(mozilla::wr::WrWindowId, bool)
mozilla::detail::RunnableMethodImpl<RefPtr<mozilla::layers::APZCTreeManager>,void (mozilla::layers::IAPZCTreeManager::*)(unsigned long long,
New profile:
https://bit.ly/2y8UohW
|
|
||
Comment 41•4 years ago
|
||
|
|
||
Comment 42•4 years ago
|
||
Extra info:
Device information attached, summary:
Processor Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz, 3096 Mhz, 14 Core(s), 28 Logical Processor(s)
GPU NVIDIA GeForce RTX 2060
Edge(Chromium) is smooth.
|
|
Assignee | |
Updated•4 years ago
|
|
|
Assignee | |
Comment 43•4 years ago
|
||
Looking at the GPUview logs, one thing that stands out is that we every time we drop a frame we have a long video decode packet (15ms). Edgium never has video decode packets that take longer than 10ms.
jya, do you know any reason we'd have different behaviour here than Chrome/Edgium?
|
||
Comment 44•4 years ago
|
||
Is VP9 HW decoding enabled (can be checked with the media devtools installed, press ctrl-shift-I, go to the media tab, select the video and check the name of the video decoder in use)
Is the stutter occurring in the same spot in the video, or is it every 30s regardless of where you start from?
I noticed one around 1m (it's very subtle)
Can you tell if frames are dropped because we are slow, or it's just frames are timed that way?
If it's happening in particular point, it could be the timestamps calculated are wrong, this is where our code could differ with Chrome/Edge.
Chrome/Edge use libvpx when software decoding and we use ffmpeg ; unless it's the WMF hardware decoder. Difference could be there too. So need to confirm the decoder in use.
As for pure performance; the only thing coming to mind is that we perform an extra copy out of the decoder into a new (or re-used) surface.
https://searchfox.org/mozilla-central/source/dom/media/platforms/wmf/DXVA2Manager.cpp#911
Most times I've seen hold-up were due to that copy.
Chrome/new Edge do the decoding in the compositor process, and have removed all copies. This is something I believe we should be doing too.
|
|
Assignee | |
Comment 45•4 years ago
|
||
Yep, this is with VP9 HW decoding enabled in both browsers.
I think the extra copies would show up in the 3d hw pipeline and not the video hw pipeline which suggests that it's not that.
|
|
||
Comment 46•4 years ago
|
||
Are those stutters only happening with this video or all?
Playing with PlatformDecoderModule:5 may show an issue with how the timestamps are calculated.
However, that would have no impact on how long it takes to decode.
Another thing maybe is that we keep a pretty long queue of decoded frame in both the MediaDecoderStateMachine and in the compositor itself.
There could be over 10 frames, something to do with how we re-use/re-cycle our image when doing the copy?
Reading the earlier posts (should have done that earlier I guess), seems that HW decoding or not makes no difference in the output, which would remove the decoder from the list of suspects.
running with MediaSource:5,MediaSourceSamples:5 will show the raw timestamps as found in the webm container. Would be interesting to see if there's anything going on around the time where the stutter appear.
|
|
||
Updated•4 years ago
|
|
|
Reporter | |
Comment 47•4 years ago
|
||
(In reply to Jean-Yves Avenard [:jya] from bug #1634991 comment #0)
I noticed that none of my machines with VP9 hardware decoding capabilities are now using the vp9 HW decoder.
Only ffvp9 appears to be used (you can verify with the media devtools add-on).
This leads to higher CPU usage than needed and lower performance.
Are you referencing this addon: https://addons.mozilla.org/en-US/firefox/addon/devtools-media-panel/ ? If not, can you post a link to the addon you were using so I can test myself on the referenced URL to see what capabilities are being utilized? With the current addon I do get "No media" on the YouTube video and thus can not test it, however, on manual 1080p after the video has been fully buffered the difference in total CPU/GPU usage based on Windows's 10 taskmanager between the video being paused and playing is ~0.5% CPU/0% GPU usage and 5% CPU/~16.5% GPU usage (AMD Phenom II X6 1045T/GeForce GTX 1650).
|
|
||
Comment 48•4 years ago
|
||
(In reply to sworddragon2 from comment #47)
(In reply to Jean-Yves Avenard [:jya] from bug #1634991 comment #0)
I noticed that none of my machines with VP9 hardware decoding capabilities are now using the vp9 HW decoder.
Only ffvp9 appears to be used (you can verify with the media devtools add-on).
This leads to higher CPU usage than needed and lower performance.
Are you referencing this addon: https://addons.mozilla.org/en-US/firefox/addon/devtools-media-panel/ ? If not, can you post a link to the addon you were using so I can test myself on the referenced URL to see what capabilities are being utilized? With the current addon I do get "No media" on the YouTube video and thus can not test it, however, on manual 1080p after the video has been fully buffered the difference in total CPU/GPU usage based on Windows's 10 taskmanager between the video being paused and playing is ~0.5% CPU/0% GPU usage and 5% CPU/~16.5% GPU usage (AMD Phenom II X6 1045T/GeForce GTX 1650).
You can use the devtools media panel and when playing a video, press Ctrl-Shift-I , select Media-Webrtc select the video playing and expand ; look at the name of the video decoder.
I find that if sometimes a video isn't listed; by clicking on the console tab and then going back to the Media-Webrtc ; it will then show.
Otherwise, with a tool like DXVA Checker, you can see if HW decoding is being used by a particular process.
|
|
Reporter | |
Comment 49•4 years ago
|
||
(In reply to Jean-Yves Avenard [:jya] from comment #48)
You can use the devtools media panel and when playing a video, press Ctrl-Shift-I , select Media-Webrtc select the video playing and expand ; look at the name of the video decoder.
I find that if sometimes a video isn't listed; by clicking on the console tab and then going back to the Media-Webrtc ; it will then show.
Unfortunately nothing of this does work here. No matter what I additionally try it does never find any media. I did even make a test on Twitch but no media can't be found there too. But based on the comments of the addon I'm not alone with such troubles as 1 and 2 years ago there are similar reports.
(In reply to Jean-Yves Avenard [:jya] from comment #48)
Otherwise, with a tool like DXVA Checker, you can see if HW decoding is being used by a particular process.
With DXVA Checker 4.3.0 on playing the YouTube video on manual 1080p the tab "GPU Engine Usage" shows percentages for 2 processes:
3D VideoDecode Copy 1
dwm ~4.5% ~0.1%
firefox ~11% ~10% ~0.1%
So hardware video decoding capabilities appear to be successfully being utilized. But I'm still experiencing the microstuttering on an up-to-date system which implies it is not due to the hardware video decoder not being utilized (bug #1634991).
|
|
Reporter | |
Comment 50•4 years ago
|
||
Although the last profile is from :ktaeleman I guess I give some informations to comment #44 and comment #46 based on the bug report (in case they are still needed and not already redundant). But since those answers might be required from the profile I'm still requesting a needinfo to :ktaeleman and :jrmuizel also since it looks to me some questions are still open for those 2 comments (if nothing is left feel free to cancel):
(In reply to Jean-Yves Avenard [:jya] from comment #44)
Is VP9 HW decoding enabled (can be checked with the media devtools installed, press ctrl-shift-I, go to the media tab, select the video and check the name of the video decoder in use)
Based on comment #49 hardware decoding is being utilized.
(In reply to Jean-Yves Avenard [:jya] from comment #44)
Is the stutter occurring in the same spot in the video, or is it every 30s regardless of where you start from?
The latter is the case.
(In reply to Jean-Yves Avenard [:jya] from comment #46)
Are those stutters only happening with this video or all?
Not with all but roughly estimated with 20%-25% of all videos that I see on YouTube even accross different random users and categories.
@:jrmuizel: Otherwise how can this ticket progress further? So far this issue seems to be reproducible and we have a profile, is there anything specific we could do now?
|
|
Assignee | |
Comment 51•4 years ago
|
||
A lot has happened and changed over the course of this bug. I want to make sure we're able to see what you're seeing are you able to take a video of the problem happening with a phone or something like that and share it here?
|
|
Reporter | |
Comment 52•4 years ago
|
||
(In reply to Jeff Muizelaar [:jrmuizel] from comment #51)
A lot has happened and changed over the course of this bug. I want to make sure we're able to see what you're seeing are you able to take a video of the problem happening with a phone or something like that and share it here?
As rare as it probably is I do not even own a smartphone or any other external video capturing device and borrowing one is also not a possibility here. Any other ideas how to solve this? The only alternatives for now I could think of would be:
-
The simplified reproduction steps from comment #29 (which I will also add at the end of this post [1]) make it easy to see this issue even on very unlucky scenarios in about 2-3 minutes. In case checking this out yourself would give an idea what is happening here and how it could be solved.
-
Since :ktaeleman can also reproduce this issue maybe he can provide a video capture if he has the capabilities like a smartphone which I lack (or other more professional capabilities like a direct video capture - if this would not risk obfuscating the issue via some Win-API/driver internal stuff or similar things).
[1] With the current release of Firefox (version 76.0.1 - as newer versions might apply at default the new VSYNC-logic and slightly obfuscate this issue) go to https://www.youtube.com/watch?v=FYCjJHhxONc (the video from the URL field), set the video quality manually to 1080p (not strictly required - but for potential better consistent reproducibility) and repeat the video from ~1:56 to ~2:01 until the microstuttering is happening (which will be very obvious). I usually need around ~2-6 tries to get one so doing a few more on unlucky cases should be safe.
|
|
Assignee | |
Comment 53•4 years ago
|
||
(In reply to sworddragon2 from comment #52)
- The simplified reproduction steps from comment #29 (which I will also add at the end of this post [1]) make it easy to see this issue even on very unlucky scenarios in about 2-3 minutes. In case checking this out yourself would give an idea what is happening here and how it could be solved.
To confirm, do you see this in fullscreen mode or windowed mode?
|
|
Assignee | |
Updated•4 years ago
|
|
|
Reporter | |
Comment 54•4 years ago
|
||
I always do the tests on a maximized (but non-fullscreen via F11) Firefox window with the YouTube player not being in fullscreen mode. However, this issue does also reproduce if the YouTube player is in fullscreen mode.
|
|
Assignee | |
Comment 55•4 years ago
|
||
sworddragon2, can you try disabling hardware video decoding by setting media.hardware-video-decoding.failed = true and see if that helps?
|
|
Reporter | |
Comment 56•4 years ago
|
||
Setting media.hardware-video-decoding.failed to true, restarting the browser and testing again causes the microstuttering still to reproduce apparently even unchanged.
|
|
Assignee | |
Comment 57•4 years ago
|
||
(In reply to sworddragon2 from comment #56)
Setting media.hardware-video-decoding.failed to true, restarting the browser and testing again causes the microstuttering still to reproduce apparently even unchanged.
Ah sorry I suggested the wrong pref. I meant set media.hardware-video-decoding.enabled=false
|
|
Reporter | |
Comment 58•4 years ago
|
||
Even setting media.hardware-video-decoding.enabled to false and restarting the browser does not cause the issue to change. The microstuttering still appears as usual.
|
|
Assignee | |
Updated•4 years ago
|
|
|
||
Comment 59•4 years ago
|
||
Hey Kris - can you see if you reproduce this again?
|
|
||
Comment 60•4 years ago
|
||
Yes, I can still easily reproduce this.
However, for me the stuttering disappears when disabling media.hardware-video-decoding.enabled.
I tried a couple of times enabled/disabled and the results where consistent.
media.hardware-video-decoding.enabled false -> None to very minimal stuttering
media.hardware-video-decoding.enabled true -> Constant microstuttering
|
|
Assignee | |
Comment 61•4 years ago
|
||
Kris, do you even see stuttering on https://www.youtube.com/watch?v=FYCjJHhxONc at 1080p?
|
|
||
Comment 62•4 years ago
|
||
@jrmuizel:
That video is not very smooth to begin with though, the panning stutter feels the same in both firefox and chromium.
I did have a 1 short stutter ~5seconds in that video but no microstuttering as far as I can tell.
Comment 60 sounds like bug 1638405.
|
|
Reporter | |
Comment 64•4 years ago
|
||
Since Kris might be actually encountering another bug and eventually not be able to reproduce this issue I have tried to get another profile (as the last time I tried that it failed at some point). This time I was able to capture a profile, the details are:
- This was being tested on Firefox 76.0.1 with a complete new profile without changing any special settings (except maximizing the window).
- Firefox has been restarted once after the new profile has been created.
- The issue has been tested at https://www.youtube.com/watch?v=FYCjJHhxONc (with being the only open tab) by disabling YouTube's autoplay option, setting the video manually to 1080p and fully prebuffering the video.
- A few minutes have been passed before making the tests to avoid potential noises from eventual initial background tasks from the new profile.
- The Media preset have been choosen for the profiler.
- A few tries of capturing have been done (to confirm that I see multiple significant microstutterings in the final capture). Before starting the next potential final capture some seconds have been passed to make sure there are not too much noises from the GC.
The link to the capture is https://share.firefox.dev/2BnQIdD . @jrmuizel: Is there anything suspicious in the profile that could explain the microstuttering?
|
|
Assignee | |
Comment 65•4 years ago
|
||
I don't see anything obvious from that profile, but gecko profiles aren't that easy to see stuttering in. Let's try a different approach:
Can you get a GPUView trace of the problem happening? https://docs.microsoft.com/en-us/windows-hardware/drivers/display/using-gpuview?
You should be able to run log.cmd (as administrator) to start the logging, cause the stuttering, and run log.cmd again to stop the logging. This will produce a "Merged.etl" file. It will probably be pretty big but you should be able to post a link to it on Google drive or something like that.
This should give us a good overview of what's going wrong and confirm whether anyone who reproduces it is seeing the same thing.
|
|
Reporter | |
Comment 66•4 years ago
|
||
I did install the Windows Performance Toolkit via the Windows Assessment and Deployment Kit for Windows 10 version 2004, opened a command prompt as administrator and executed log.cmd as a test if all works fine. The command could not be found (as I guess the PATH variable has not been gotten extended) so I switched to "D:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\gpuview" and executed log.cmd. It has done some initialization steps but aborted after a few seconds with the message '"4000" kann syntaktisch an dieser Stelle nicht verarbeitet werden.' which very roughly translates to '"4000" can't be processed syntactically on this position.'. I tried to get an accurate english translation (and an eventual solution) via Google but did not found any entry. The german variant did find 1 entry but it did not yield a solution.
It looks like a GPUView trace can't be done with that issue (except somebody would have a solution). Any other ideas how to continue now with this ticket?
|
|
Assignee | |
Updated•4 years ago
|
|
|
||
Updated•4 years ago
|
|
|
||
Comment 67•4 years ago
|
||
@sworddragon2: Looks like this is an issue with GPUView not supporting your locale properly:
https://www.yosoygames.com.ar/wp/2016/09/solving-gpuview-4000-was-unexpected-at-this-time-error/
Could you try the solution mentioned in the above link?
|
|
Reporter | |
Comment 68•4 years ago
|
||
|
|
Reporter | |
Comment 69•4 years ago
|
||
I was able to fix the issue and got a GPUView trace. Also I did some tests[1] before getting the trace as I just updated yesterday to Firefox 77.0.1.
The second random filehoster I found via Google did work on uploading the trace. However, on pasting the download link into the file field - as Bugzilla exposes it as one of multiple options - instead of forwarding the download link to the filehoster the pasted link is downloaded as binary file. It appears I can not fix it by editing the attachment but opening the 61 bytes small file in an editor (for example Notepad) will show the real download link. The trace is compressed with 7-Zip on high compression settings and has a compressed size of 145 MiB (2.7 GiB uncompressed). On getting the trace I did the same steps as on getting a profile a few days ago (new Firefox profile; maximized window; restarted Firefox at least once; disabling autoplay on Youtube and choosing manually 1080p; prebuffering the video; waiting a few minutes before starting the trace logging) and additionally restarted Windows shortly before. The trace contains the information from playing the complete video ( https://www.youtube.com/watch?v=FYCjJHhxONc ) once while I noticed the microstuttering a few times.
[1] As expected it appears the setting gfx.vsync.force-disable-waitforvblank (defaulting to false) enables the new VSync logic at default for Firefox 77. The microstuttering still exists but looks quite more smooth (as already observed in an earlier test). I did also make another test by setting gfx.webrender.force-disabled to true again and the video has then no microstuttering anymore but a few very minor single stutters which are better than I remember from the past test (probably due to the new VSync logic) but on a very short test with Twitch the streams are still a bit derping out (I'm very unsure what it actually is - I think it looked a bit like a sort of trailing of frames). While enabling Webrender significantly decreases the visible performance here on YouTube I'm still a bit curious if the source of this issue is actually Webrender (and thus the minor stutters with disabled Webrender are normal - eventually due to a more or less outdated renderer now in use) or if something else is the cause the microstuttering (possibly including the minor stutters with disabled Webrender).
|
||
Comment 70•4 years ago
•
|
||
Comment on attachment 9154373 [details]
GPUView trace
Download URL: https://gofile.io/d/2jYGFr
|
|
Assignee | |
Comment 71•4 years ago
|
||
I had an initial look at the trace. I see places in the trace that look microstuttering. There's no evidence that the system is under high load, instead it looks like our timing is just bad. I'll continue to look to see if I can tease out why.
sworddragon2, just to confirm, was this trace taken with gfx.vsync.force-disable-waitforvblank set to true or to false?
|
|
Reporter | |
Comment 72•4 years ago
|
||
(In reply to Jeff Muizelaar [:jrmuizel] from comment #71)
sworddragon2, just to confirm, was this trace taken with gfx.vsync.force-disable-waitforvblank set to true or to false?
The trace was being taken with a new profile on Firefox 77.0.1 without altering any special settings. In this case it means gfx.vsync.force-disable-waitforvblank was on its default value (false).
|
|
Assignee | |
Comment 73•4 years ago
|
||
I've looked at this some more. It looks like there's an issue where the timestamps of the frames aren't in sync with the timestamps we use for composition and we jitter them back into sync.
|
|
Assignee | |
Comment 74•4 years ago
|
||
The composition time that we use looks like it comes from 3 places:
https://searchfox.org/mozilla-central/rev/0e09b9191c02097034e46b193930f91c45b7885d/gfx/layers/wr/WebRenderBridgeParent.cpp#1129
https://searchfox.org/mozilla-central/rev/0e09b9191c02097034e46b193930f91c45b7885d/gfx/layers/wr/WebRenderBridgeParent.cpp#1199
https://searchfox.org/mozilla-central/rev/0e09b9191c02097034e46b193930f91c45b7885d/gfx/layers/wr/WebRenderBridgeParent.cpp#2065
Video frames get their times assigned here:
https://searchfox.org/mozilla-central/rev/0e09b9191c02097034e46b193930f91c45b7885d/dom/media/mediasink/VideoSink.cpp#453
I suspect we need to be more disciplined in keeping these two places synced up in order to avoid the jitter.
Jean-Yves, what are media's expectations for how this is supposed to work? Can we choose time stamps so that they're in the middle of a vsync and not close to the edge so that we're less likely to stutter?
|
|
||
Comment 75•4 years ago
|
||
(In reply to Jeff Muizelaar [:jrmuizel] from comment #74)
The composition time that we use looks like it comes from 3 places:
https://searchfox.org/mozilla-central/rev/0e09b9191c02097034e46b193930f91c45b7885d/gfx/layers/wr/WebRenderBridgeParent.cpp#1129
https://searchfox.org/mozilla-central/rev/0e09b9191c02097034e46b193930f91c45b7885d/gfx/layers/wr/WebRenderBridgeParent.cpp#1199
https://searchfox.org/mozilla-central/rev/0e09b9191c02097034e46b193930f91c45b7885d/gfx/layers/wr/WebRenderBridgeParent.cpp#2065Video frames get their times assigned here:
https://searchfox.org/mozilla-central/rev/0e09b9191c02097034e46b193930f91c45b7885d/dom/media/mediasink/VideoSink.cpp#453I suspect we need to be more disciplined in keeping these two places synced up in order to avoid the jitter.
Jean-Yves, what are media's expectations for how this is supposed to work? Can we choose time stamps so that they're in the middle of a vsync and not close to the edge so that we're less likely to stutter?
I'm unfamiliar with that code and I'm a bit baffled it would be there.
Rewriting bad timestamps is one thing, doing this all the time is another. The decoder should only return timestamp has found in the container.
IMHO, I don't think you want the decoder/videosink to do anything else but provide the information the video itself said when they should be presented.
Up to the compositor/rendering to do whatever is needed.
So whatever fixup is being done in the videosink should be removed and moved to the gfx code.
ni? :alwu has he made those last changes.
I'm not sure this is what's causing this actual problem though. Those changes were done last year, and this bug is 4 months old.
It is certainly possible that with HW decoding the timestamps set by the windows decoder are wrong for some streams; it wouldn't be the first time it happens. Never seen it with YouTube video however; as they are very good at packaging their videos.
|
|
Assignee | |
Comment 76•4 years ago
|
||
To rephrase my question: What is the starting timestamp that video frames have their timestamps offset from?
e.g.
(assuming 60fps is exactly 15ms)
If we get vsyncs every 15ms at starting at 0ms and then at 15ms, 30ms, 45ms, etc. . It makes a difference whether a 60fps video has it's frames timestamps at 0ms, 15ms, 30ms, 45ms. vs 5ms, 20ms, 35ms, 50ms. Starting with 0ms we're very susceptible to timing jitter causing us to display a frame late or early. Starting with 5ms we have a lot more room for jitter without causing an early or late frame.
|
||
Comment 77•4 years ago
|
||
(In reply to Jeff Muizelaar [:jrmuizel] from comment #76)
To rephrase my question: What is the starting timestamp that video frames have their timestamps offset from?
The media samples would start from zero. Take [1] (from comment64) as an example, the first audio frame is [0, 13500] (us) and the first video is [0, 33000] (us).
[1] https://www.youtube.com/watch?v=FYCjJHhxONc
I've looked at this some more. It looks like there's an issue where the timestamps of the frames aren't in sync with the timestamps we use for composition and we jitter them back into sync.
Because I'm not familiar how we render the video frame, could you explain this a little bit more to me?
Thank you.
|
|
Assignee | |
Comment 78•4 years ago
|
||
VideoSink::RenderVideoFrames() associates a mozilla::TimeStamp with a video frame. Walking up a little bit it looks like this timestamp is based off of a TimeStamp::Now() when AudioSinkWrapper::GetPosition(TimeStamp* aTimeStamp) is called which is then adjusted based on how far along we are in audio playback/media clock.
So this leads to a couple of questions:
- When does VideoSink::RenderVideoFrames()/AudioSinkWrapper::GetPosition get called? (How does this compare to vsync)
- When does the audio playback/media clock start relative to vsync.
- Does it make sense for video frames to be timestamped based on when VideoSink::RenderVideoFrames()/AudioSinkWrapper::GetPosition runs?
|
|
||
Comment 79•4 years ago
|
||
The process of sending video frames to compositor is like following.
We would inregularly send multiple video frames to the compostior, [1] which start time all exceed the current audio time. We can call them "current frame + future frames". Each video frame has its timestamp, for those futrue video frames, we would calculate its estimated timestamp [2] based on the time difference from the current audio time and the audio playback rate.
eg. if the video frame is for 15.5s, but audio time is 15.2s. So the video's timestamp would be [TimeStamp::Now() + (15.5 - 15.2)/(audio playback rate)].
All those video frames would eventually be stored in the the ImageComposite [3]. When someone is asking it for an image, it would check the composition time [4], which is probably the one you mentioned in the comment 74, in order to select the current image [5] which start time exceeds the current composition time.
[1] https://searchfox.org/mozilla-central/rev/5e6c7717255ca9638b2856c2b2058919aec1d21d/dom/media/mediasink/VideoSink.cpp#471
[2] https://searchfox.org/mozilla-central/rev/5e6c7717255ca9638b2856c2b2058919aec1d21d/dom/media/mediasink/VideoSink.cpp#439-441
[3] https://searchfox.org/mozilla-central/rev/5e6c7717255ca9638b2856c2b2058919aec1d21d/gfx/layers/composite/ImageComposite.cpp#210
[4] https://searchfox.org/mozilla-central/rev/5e6c7717255ca9638b2856c2b2058919aec1d21d/gfx/layers/composite/ImageComposite.cpp#93
[5] https://searchfox.org/mozilla-central/rev/5e6c7717255ca9638b2856c2b2058919aec1d21d/gfx/layers/composite/ImageComposite.cpp#107-126
- When does VideoSink::RenderVideoFrames()/AudioSinkWrapper::GetPosition get called? (How does this compare to vsync)
If we're still in video decoding, we would check the audio clock time whenever we get a new decoded video frame, then decide if we need to send current video frames to the compositor. In addition, when we send video frames to the compositor, we would also schedule another task to run the rendering process [6] again depending on the time difference between the next video frame, which would be rendered, and the current audio time.
eg. current video frame is 15s, next video frame is 15.5, audio time is 15.2. So we would schedule a task that should be run at [TimeStamp::Now() + (15.5 - 15.2)/(audio playback rate)].
- When does the audio playback/media clock start relative to vsync.
In media code, our logic isn't related with the vsync, but the compositor would check vsync's value [7] in order to make sure that the video frame rate we provide would never exceed the vsync rate.
- Does it make sense for video frames to be timestamped based on when VideoSink::RenderVideoFrames()/AudioSinkWrapper::GetPosition runs?
I'm not sure I got the point of this question. If we don't provide the timestamp in the video sink, then how the compositor can know which image should be selected?
Thank you.
|
|
Assignee | |
Updated•4 years ago
|
|
|
Assignee | |
Comment 80•4 years ago
|
||
I looked some more at this today and noticed the timestamps on the frames seem to be an integer number of milliseconds apart and not always the same:
34, 33, 34, 33, 33, 34, 33, 34, 33, 33
Alastor, any idea why the frame timestamps aren't more accurate? i.e. separated by 33.333 milliseconds?
|
|
Assignee | |
Comment 81•4 years ago
|
||
(In reply to Jeff Muizelaar [:jrmuizel] from comment #80)
I looked some more at this today and noticed the timestamps on the frames seem to be an integer number of milliseconds apart and not always the same:
34, 33, 34, 33, 33, 34, 33, 34, 33, 33
It looks like the frame time here: https://searchfox.org/mozilla-central/source/dom/media/mediasink/VideoSink.cpp#439 is already rounded to the nearest millisecond.
|
|
Assignee | |
Comment 82•4 years ago
|
||
Hmm it seems like the rounded timestamps come directly from the video file and that only the webm file has rounded timestamp.
0.000000
0.033000
0.067000
0.100000
0.133000
0.167000
0.200000
0.234000
0.267000
0.300000
0.334000
The mp4 version of the same video has timestamps that look like:
0.000000
0.033367
0.066733
0.100100
0.133467
0.166833
0.200200
0.233567
|
|
Assignee | |
Comment 83•4 years ago
|
||
sworddragon2, do you notice any difference if you use h264ify addon to force the video to use h264?
|
|
Reporter | |
Comment 84•4 years ago
|
||
I did test in the past if requesting another codec from YouTube does have any different effect. But instead of using an addon I disabled WebM directly in Firefox to request H.264 on YouTube:
(In reply to sworddragon2 from comment #10)
- Setting media.webm.enabled to false changes the used codec according to YouTube's video player stats from vp09 to avc1 but the issue still exists.
|
|
||
Comment 85•4 years ago
|
||
(In reply to Jeff Muizelaar [:jrmuizel] from comment #80)
I looked some more at this today and noticed the timestamps on the frames seem to be an integer number of milliseconds apart and not always the same:
34, 33, 34, 33, 33, 34, 33, 34, 33, 33Alastor, any idea why the frame timestamps aren't more accurate? i.e. separated by 33.333 milliseconds?
What are those number for? The frame's length? And why this would cause a microstuttering?
As you mentioned in comment81, if now we haven't reach the time to play a video frame, then we would calculate a future timestamp for it.
|
||
Comment 86•4 years ago
|
||
I found a cause for wildly fluctuating video frame timestamps in bug 1649859. It might be the cause for this bug, too.
|
|
||
Comment 87•4 years ago
|
||
Or not; that particular problem does not seem to exist on Windows, at least not to the same degree.
|
|
Assignee | |
Updated•4 years ago
|
|
|
Assignee | |
Comment 88•4 years ago
|
||
Another thing I just noticed: with WebRender we call UpdateBias at 60fps when playing 30fps video. This means that biasing will always be None. With Non-WebRender we only call UpdateBias after we've actually rendered. I'll put together a patch that fixes this.
|
|
Assignee | |
Comment 89•4 years ago
|
||
sworddragon2, can you try this build https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/OWzV0QDBRBGd0dVgEwD_kQ/runs/0/artifacts/public/build/target.zip and see if it makes things better for you?
|
|
Reporter | |
Comment 90•4 years ago
|
||
Is this a nightly build with the patch from bug #1652181 ? Testing this build does cause the microstuttering to change: Instead of getting a microstuttering every ~15-30 seconds that lasts very roughly for ~0.5-2 seconds I get every second ~1-3+ short single stutters. It is probably an improvement since the big strikes are gone now but the performamce is still very broken.
|
|
Assignee | |
Comment 91•4 years ago
|
||
Ok, bug 1652181 is landed so I'm glad to hear that it helps some. Based on your feedback I had a closer look at a GPUview trace that I did locally and every 12.5 seconds or so I see a 1.6s period of negative bias and then a skipped frame. I'll see if I can figure out what's causing that.
Just to confirm, does a Nightly build that includes a fix for bug 1652181 still have worse stuttering with WebRender on vs off?
|
|
Reporter | |
Comment 92•4 years ago
|
||
On testing the current Nightly with a new profile:
-
With Webrender enabled (gfx.webrender.force-disabled set to false): The stuttering is similar to that of comment #90. However, for some reason it appears less frequently (roughly 1-2 stutters every 3-5 seconds). I would consider this now being a drastic improvement compared to before the state of bug #1652181 .
-
With Webrender disabled (gfx.webrender.force-disabled set to true): The stuttering is very similar to that of Webrender being enabled. The frequency of the stuttering is the same but the type of the stuttering is randomly either a single stutter or a very short series of microstuttering. On comparing this to comment #14 I'm unsure if this is an improvement over time compared to the estimated 10%-20% at this time (I think it is a slight improvement). Out of curiosity on testing a stream on Twitch they still derp out.
This means the Webrender performance has hugely improved and is now even slightly better than Non-Webrender. I think it is currently equal or better than any workaround that I have discovered in the past without having the disadvantages of those workarounds (like Twitch streams becomming sort of fuzzy/streaky). The performance is close to being acceptable but it still looks not fine to me (and is significantly worse than Google Chrome was in my test months ago).
2 things I'm a bit curious about is:
- The negative bias/skipped frame you found.
- The progress of bug #1649859 . However, comment #87 makes it not fully clear to me if there might be a slight improvement on Windows.
Eventually progress in any of these does cause any other improvement - maybe enough to fix this issue. If not I'm a bit curious about if the goal of this ticket is mainly to get a performance similar to non-Webrender or to (nearly) fully eliminate the stuttering to get a basically flawless experience that would be normal? Since we probably don't know for sure if all of the remaining stuttering is comming now from the Webrender path or if it is something else.
|
|
Assignee | |
Comment 93•4 years ago
|
||
So the UpdateBias stuff is meant to adjust the the timestamps of video frames so that we don't jitter back and forth if the frame timestamps are very close to vsync. This was broken with WebRender because we were calling it a the same rate as vsync and not the frame rate of video. Bug 1652181 fixed that to so that it matches the behaviour of non-WebRender.
As far as we can tell bug 1649859 is a macOS only problem so I don't think any changes there will actually help with this bug.
Since the WebRender and non-WebRender paths now have roughly the same the same amount of stutter I think it's probably a good idea to close this bug and file a new one for the remaining problem.
|
|
Assignee | |
Comment 94•4 years ago
|
||
Also, sworddragon2, thanks a lot for all of your testing and persistence on this issue! Hopefully, we can solve the remaining problem.
|
|
Reporter | |
Comment 95•4 years ago
|
||
Thanks for your help on mitigating this issue as the microstuttering is now technically fixed and just frequent single stutters remain. I have created a new report for those on bug #1652540 and added you to the CC list.
|
|
Assignee | |
Updated•4 years ago
|
|
||
Updated•4 years ago
|
Description
•