1709486 - Youtube 8k video only watchable in troubleshooting mode (MacOS)

Status: NEW

(Core :: Graphics: WebRender, defect, P3)

Description

[john.sd.siu]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:90.0) Gecko/20100101 Firefox/90.0

Steps to reproduce:

1. Open FF normally and disable all addon manually (NOT troubleshoot mode)
2. Open URL https://www.youtube.com/watch?v=BtYKDamqo2I
3. After ads, choose 8k(4320p50) manually
4. Play video

---

5. Restart FF with troubleshoot mode
6. Open URL https://www.youtube.com/watch?v=BtYKDamqo2I
7. After ads, choose 8k(4320p50) manually.
8. Play video

Actual results:

At step (4) video will play for >1sec and picture stop for 5-7sec, while the audio continue normally. Then the picture play for a split sec again and stop for a few seconds. The cycles keep going.

At step (8) video play normally.

Expected results:

Step (4) should play 8k video normally

PS:

• Same happen on nightly (90)
• Tested with a complete new profile(no addon, no about:config change, straight to the url), same behavior, 8k play normally only in troubleshoot mode
• Tested with a few other 8k videos on youtube, same behavior
• Tested with Chrome, same videos played normally

Comment 1

[john.sd.siu]

Typo correction for "Actual results" 1st sentence:

At step (4) video will play for <1sec and picture stop for 5-7sec, ...

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 3

[john.sd.siu]

Attachment: slow mo guys first 14sec non-troubleshoot mode

Comment 4

[john.sd.siu]

Attachment: slow mo guys first 16sec troubleshoot mode

Comment 5

[john.sd.siu]

Attachment: slow mo guys run till the end in Chrome

Comment 6

[john.sd.siu]

Attachment: slow mo guys first 16sec in safari

Comment 7

[john.sd.siu]

System Info:
OS: macOS Big Sur 11.3.1
CPU: 3.2 GHz 8-Core Intel Xeon W
GPU: Radeon Pro Vega 56 8 GB

Maybe this can be break down into 2 issues:

1. Why troubleshoot mode perform better(like double?!)?
2. Why FF is dropping ~40%(troubleshoot mode) / ~90%(non-troubleshoot) the frames? While Chrome and Safari like none.

Comment 8

[C.M.Chang[:chunmin]]

The dropped frames of https://www.youtube.com/watch?v=BtYKDamqo2I in 8k quality are roughly equal for normal mode and troubleshoot mode on my Ubuntu 20.04 (kernel 5.8.0-50-generic) with NVIDIA Corporation GP107GL [Quadro P400]. Both are around 400 (dropped frames) / 1200 (total frames).

Would you mind sharing your about:support information? Do you change any settings in your about:config page?

Comment 9

[C.M.Chang[:chunmin]]

Bryce, any idea what's going on, or who might know the problem better? Graphic team perhaps?

Comment 10

[john.sd.siu]

(In reply to C.M.Chang[:chunmin] from comment #8)

The dropped frames of https://www.youtube.com/watch?v=BtYKDamqo2I in 8k quality are roughly equal for normal mode and troubleshoot mode on my Ubuntu 20.04 (kernel 5.8.0-50-generic) with NVIDIA Corporation GP107GL [Quadro P400]. Both are around 400 (dropped frames) / 1200 (total frames).

Would you mind sharing your about:support information? Do you change any settings in your about:config page?

I just retested with a clean profile on 90.0a1 (2021-05-10) and get the same result. So no about:config change required. If you are not testing on Mac hardware, at least test with AMD gpu.

How to share about:support?

Comment 11

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

(In reply to C.M.Chang[:chunmin] from comment #9)

Bryce, any idea what's going on, or who might know the problem better? Graphic team perhaps?

Let's gather some more information and see if we can figure out where the frames are being dropped. If it's during composition, gfx folks sound suitable, but it's possible we're not decoding fast enough or seeing another issue in the media pipeline.

How to share about:support?

If you navigate to about:support that page should have two buttons to copy that data, if you use either of those to copy the data then attach it to this bug.

Could you also try using the Firefox Profiler and use the drop down box to select the 'Media' preset, then capture a profile of the issue taking place, upload the profile with the button in the profiler, and share the profile on this bug?

Comment 12

[john.sd.siu]

Attachment: about:config

(In reply to Bryce Seager van Dyk (:bryce) from comment #11)
> (In reply to C.M.Chang[:chunmin] from comment #9)
> > Bryce, any idea what's going on, or who might know the problem better? Graphic team perhaps?
> 
> Let's gather some more information and see if we can figure out where the frames are being dropped. If it's during composition, gfx folks sound suitable, but it's possible we're not decoding fast enough or seeing another issue in the media pipeline.
> 
> > How to share about:support?
> 
> If you navigate to `about:support` that page should have two buttons to copy that data, if you use either of those to copy the data then attach it to this bug.
> 
> Could you also try using the [Firefox Profiler](https://profiler.firefox.com/) and use the drop down box to select the 'Media' preset, then capture a profile of the issue taking place, upload the profile with the button in the profiler, and share the profile on this bug?

Comment 13

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Could you also capture a Firefox profiler run of the issue taking place and upload it and share the link?

Comment 14

[john.sd.siu]

(In reply to Bryce Seager van Dyk (:bryce) from comment #13)

Could you also capture a Firefox profiler run of the issue taking place and upload it and share the link?

Following are Firefox nightly 90.0a1(2021-05-12)) profiling with media preset in different mode when watching first ~25sec of https://www.youtube.com/watch?v=BtYKDamqo2I.

normal mode : https://share.firefox.dev/33JaF9w
troubleshoot : https://share.firefox.dev/3y9gcV6

Comment 15

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

I initially thought this could be due to different decoders being used in troubleshooting mode, since troubleshooting disables hardware decoding. However, both profiles appear to be using software decoding, so it's unlikely to be that.

However, there is a difference in the image format being used. In normal mode we use a MacIOSurfaceImage to hold the video, while in troubleshooting we use a SharedPlanarYCbCrImage. Seems like that functionality is webrender related.

Reporter, could you check if setting the pref gfx.webrender.force-disabled to true, restarting the browser, and retesting in normal mode has any impact? You can set the pref back to false after testing.

Comment 16

[john.sd.siu]

(In reply to Bryce Seager van Dyk (:bryce) from comment #15)

Reporter, could you check if setting the pref gfx.webrender.force-disabled to true, restarting the browser, and retesting in normal mode has any impact? You can set the pref back to false after testing.

After changing the setting to true and restart browser, the same video can be play in 8k with no drop frame for >1min (It dropped 7frames during switching to full screen mode). Attaching the screenshot below.

Comment 17

[john.sd.siu]

Attachment: Screen Shot 2021-05-27 at 12.58.59 PM.png

Comment 18

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Thanks for checking that!

I'm moving this to graphics based on the above. Gfx folks, ping me or move back to media as necessary.

Comment 19

[Jim Mathies [:jimm]]

• Decoding is taking a very long time
• Copy into an IO Surface is slower (WebRender specific? Basic layers may be doing something else)
• MediaStateMachine collects frames and discards old frames
• MSM push the set of decoded frames over to the compositor
• Missing a marker when the compositor drops a frame in the last case

Comment 20

[Brad Werth [:bradwerth]]

In addition to finding some way to decode faster, the logic in VideoSink::UpdateRenderedVideoFrames may be relevant to do something "smarter" when we can't decode a frame in time. Users will likely want to see janky frames instead of no frames.

Comment 21

[Jim Mathies [:jimm]]

(In reply to Brad Werth [:bradwerth] from comment #20)

In addition to finding some way to decode faster, the logic in VideoSink::UpdateRenderedVideoFrames may be relevant to do something "smarter" when we can't decode a frame in time. Users will likely want to see janky frames instead of no frames.

Media Playback is currently chatting a bit about changing behavior here.

Comment 22

[Brad Werth [:bradwerth]]

I'm looking at this again. VideoSink can get overwhelmed -- dropping all frames because they are too late -- but there's no communication mechanism to get that information back to the decoder. If we could, we could modify the decoder to be less aggressive. VTDecompressionSession even offers a set of quality of service tiers for precisely this purpose.

Comment 23

[Brad Werth [:bradwerth]]

(In reply to Brad Werth [:bradwerth] from comment #22)

We could modify the decoder to be less aggressive. VTDecompressionSession even offers a set of quality of service tiers for precisely this purpose.

This didn't help. In a local patch, I changed the VTDecompressionSession to set its lowest-reported level of service, and it had no effect at all. Looks like we'll need to intervene within the MediaDecoderStateMachine.