1734854 - Flickering while watching full screen live YouTube streams

Status: RESOLVED FIXED

(Core :: Graphics: Layers, defect, P1)

Description

[Andrew McCreight [:mccr8]]

For a week or two, I've noticed some flickering when watching live YouTube streams full screen on my OSX machine. It happens every 3 or 4 seconds.

I only see this on live streams (I think I may have seen it on two different ones, but I could be misremembering). I only see it when full screen. The flickering also goes away if I bring up the "stats for nerds" box.

Comment 1

[John Lin [:jhlin][:jolin]]

It sounds like a gfx related issue that happens only when a fullscreen video layer is presented. Moving to the gfx component to seek help.

Comment 2

[Andrew McCreight [:mccr8]]

[Tracking Requested - why for this release]: flickering under certain conditions on YouTube on OSX

Comment 3

[Ryan VanderMeulen [:RyanVM]]

Could be related to bug 1653417 and friends. A regression range would be helpful if possible.

Comment 4

[Brad Werth [:bradwerth]]

Sounds likely. Live YT video may be toggling the heuristic of whether or not we "specialize" video layers. If that heuristic were to change its value during a stream, it would cause layers to be unnecessarily recreated, which would manifest as flickering. I'll see if I can replicate with an arbitrary YT live stream.

Comment 5

[Brad Werth [:bradwerth]]

I can replicate. Using the Quartz Debug tool, I can see that during the live stream, we are entering and exiting the detached mode every so often. I'll see if I can make the heuristic resilient to these momentary changes.

Comment 6

[Markus Stange [:mstange]]

How exactly does it flicker? Does it show all black for a frame?

Comment 7

[Andrew McCreight [:mccr8]]

(In reply to Markus Stange [:mstange] from comment #6)

How exactly does it flicker? Does it show all black for a frame?

It is kind of like a flash of white, not quite evenly across the screen.

Comment 8

[Markus Stange [:mstange]]

That sounds really distracting.

I think we need to get to the bottom of why the flicker happens if we switch between detached and non-detached. Such switching is expected to happen under normal operation, for example when subtitles appear or disappear. So simply adjusting our heuristic won't be enough.

Comment 9

[Markus Stange [:mstange]]

Using Quartz Debug I can confirm that YouTube Live is toggling between detached and non-detached mode on my M1 MBP, but there's no flicker associated with it on this machine.

Comment 10

[Jeff Muizelaar [:jrmuizel]]

Andrew, what macOS hardware and version are you using?

Comment 11

[Andrew McCreight [:mccr8]]

MacBook Pro (16-inch, 2019), Intel UHD Graphics 630 1536 MB. I'm on 11.6.

Comment 12

[Andrew McCreight [:mccr8]]

I only get the flicker if any kind of overlays like closed captioning or (as I said above) stats for nerds are not showing.

I've finished bisection of m-c, which shows this range: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=4eda9eb8926bdd50f4b80128ce3475eb7c6d9a4d&tochange=64bfebfc3f6c63388d8a27b2a0c91020dc859666

Bug 1732230 and bug 1732367 look possibly related.

Comment 13

[Andrew McCreight [:mccr8]]

The STR I've been using now is, go to this URL: https://www.youtube.com/watch?v=iBmUjyHla3U

That's a live freed from the space station, so hopefully it'll always be up.

Then start playing the stream and hit the "f" key to go full screen. Then wait for it to hit the maximum resolution (which I guess is 1080p). This takes a little bit, and in my experience it increases the resolution twice before it hits the max resolution. If the space station is in the night part of the planet, than the bulk of the image will be black, with a few scattered colored dots. That part of the screen does not exhibit flickering for me, but the small strip at the bottom that shows the map and other information will still flicker.

Comment 14

[Brad Werth [:bradwerth]]

[:mccr8] from comment #12)

I've finished bisection of m-c, which shows this range: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=4eda9eb8926bdd50f4b80128ce3475eb7c6d9a4d&tochange=64bfebfc3f6c63388d8a27b2a0c91020dc859666

Bug 1732230 and bug 1732367 look possibly related.

mozregression shows, for me, that this has been a consistent issue since the original enabling patch in Bug 1653417. I don't think we've ever had flicker-free YoutTube streams with this feature active.

(In reply to Brad Werth [:bradwerth] from comment #5)

I can replicate. Using the Quartz Debug tool, I can see that during the live stream, we are entering and exiting the detached mode every so often. I'll see if I can make the heuristic resilient to these momentary changes.

My local testing show the heuristic is stable during these streams. We are not recreating layers during streams and therefore that is not the cause of the flicker. My best guess is that Core Media is periodically deciding that the stream frames are no longer detachable.

New theory: the EnqueueSurface function sets each frame with no timing data as a "display immediately" frame. If the frames appear too far apart, Core Media may decide the stream has ended and stops detachment. I'll try some spoofed timing information to see if that can make the stream appear continuous to Core Media.

Comment 15

[Andrew McCreight [:mccr8]]

I redid the regression range with the space station video and it ended up at bug 1731815: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=c9a191f99fa323e42bc664320d160b4ff4f3f4f1&tochange=af7e7b1d891ee07ec15c3702cbce8f509679b82a

Comment 16

[Brad Werth [:bradwerth]]

Attachment: Bug 1734854: Make NativeLayerRootCA avoid changing sublayers when it has no effect.

When isolating video layers, changes to sublayers that we are already ignoring
should not cause us to change our sublayers. This patch checks to see if the
changes to sublayers can all be safely ignored.

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ae6e0144bd5b
Make NativeLayerRootCA avoid changing sublayers when it has no effect. r=gfx-reviewers,jrmuizel

Comment 18

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/ae6e0144bd5b

Comment 19

[Ryan VanderMeulen [:RyanVM]]

Looking better for you now with today's Nightly, mccr8?

Comment 20

[Andrew McCreight [:mccr8]]

Yes. I also verified with a local build prior to landing that the patch fixed the issue for me.

Comment 21

[Ryan VanderMeulen [:RyanVM]]

Thanks for the verification! Looks like we're ready for an approval request then :)

Comment 22

[Andrew McCreight [:mccr8]]

The stream that was originally exhibiting the issue more severely is also back up today, and the issue is fixed there, too.

Comment 23

[Markus Stange [:mstange]]

Comment on attachment 9247377 [details]
Bug 1734854: Make NativeLayerRootCA avoid changing sublayers when it has no effect.

Beta/Release Uplift Approval Request

• User impact if declined: Flickering video on YouTube live streams on some Mac hardware during fullscreen playback
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: Find a machine that reproduces the flickering (this is the hardest part), log in on YouTube, watch a YouTube live stream in fullscreen mode. If the machine doesn't flicker, use Quartz Debug "Show Detached Regions" to see if the stream is toggling detached mode every so often. With this fix, it should flicker / toggle detached mode much less often.
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): The patch should be a pure optimization: It doesn't assign new layers if the assigned layers wouldn't change. It's a very local fix and I can't see much potential for damage.
• String changes made/needed:

Comment 24

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9247377 [details]
Bug 1734854: Make NativeLayerRootCA avoid changing sublayers when it has no effect.

Fixes a pretty severe issue with the macOS power optimizations shipping in 94, approved for 94.0rc1. However, we're basically out of time to spot fix any further issues which arise with this feature. We should pref off on 94 if any more come up.

Comment 25

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/79411242d4a7

Comment 26

[Timea Cernea [:tbabos][inactive]]

Hi Andrew,
Could you please also verify this on Beta?
We've tried across 4 different MacOS devices and couldn't reproduce this on the affected Nightly version. Tried with multiple Live videos, as the ones discussed on the Element chat. So we definitely can't confirm this fix if we can't reproduce it in the first place.
Thank you!

Comment 27

[Andrew McCreight [:mccr8]]

Unfortunately, I haven't been able to reproduce it either, including on a mozilla-central build I was able to reproduce it on a few days ago. The stream looks like it has a lot lower resolution than it did before, despite YouTube still saying that it is 1080p60, so maybe that has something to do with it. I've tried to find another stream that reproduces the issue but I haven't managed to yet. I'll try again some more.

Comment 28

[Brad Werth [:bradwerth]]

The reproduction of this Bug is totally dependent on what YouTube is doing "behind the scenes", either underneath the video layer or offscreen while the video is playing. If they modify the DOM to preload ads, or do analytics or load content just in case the user will scroll down to view it... all those things will trigger this Bug. Since none of it is user visible, it's going to be really hard to determine what triggers YouTube to do these changes. The good news is that the fix is narrowly designed to ignore all non-visible changes to the layers while displaying a fullscreen video, so we should have some confidence that the fix matches the problem.

Comment 29

[Andrew McCreight [:mccr8]]

I still haven't been able to reproduce this on an old m-c build that was easily reproducing it before, despite trying a bunch of streams, so I guess we can't really verify this. I guess this is at least a good sign that it isn't a common issue.