Bugzilla

Reporter

Comment 4

•

4 years ago

Alastor, I could have sworn I profiled with the media preset in my initial report, but I went ahead and profiled again: https://share.firefox.dev/2PDLD4P

Flags: needinfo?(yoasif)

Alastor Wu [:alwu]

Comment 5

•

4 years ago

Hi, when you clicked Publish, did you click Include hidden threads? Because I can't see WebContent process and its related threads on which the major media pipeline runs.

However, the weird thing is that I indeed saw MediaDecoderStateMachine thread in the list of Threads Filter in your report.

Flags: needinfo?(yoasif)

Reporter

Comment 6

•

4 years ago

I didn't uncheck anything and Include hidden threads was checked in this profile run by default (I changed nothing). Here's a new one: https://share.firefox.dev/2CfJ99A

Flags: needinfo?(yoasif)

Comment 7

•

4 years ago

•

Edited

Different playback strategies.

Chrome/Edge will pause the video/audio until it has content to play.
We favour the audio so that it is never stopped or interrupted.

How did the video sound like when played in Edge?

Few dropped frames, smooth playback (like Edge).
we have a different idea of what smooth is, that video certainly wasn't smoothed at all, 1/3rd of the frames were dropped continuously

Reporter

Comment 8

•

4 years ago

Attached video firefox-nightly_EDIT.mp4 — Details

(In reply to Jean-Yves Avenard [:jya] from comment #7)

How did the video sound like when played in Edge?

They sound the same in Firefox Nightly and Edge.

Few dropped frames, smooth playback (like Edge).
we have a different idea of what smooth is, that video certainly wasn't smoothed at all, 1/3rd of the frames were dropped continuously

Agreed that the video I took doesn't look great, but I was also playing back in both browsers and recording from within the machine.

I shut down the other apps on this machine and just played the two videos back to back as the only tabs in each browser -

Edge: 107 dropped out of 18808
Firefox: 11184 dropped out of 18721

I'm attaching video of both browsers playing back the video so you can get an idea of how they sound.

Reporter

Comment 9

•

4 years ago

Attached video edge_EDIT.mp4 — Details

Comment 10

•

4 years ago

Can you try setting media.ffvpx.enabled to false ?

this will make it use libvpx which is the same decoder as what edge/chromium is using

normally performance are much worse than with ffvp9.

Can you try disabling webrender too ?

Reporter

Comment 11

•

4 years ago

Did some additional testing:

WR disabled: 4507 dropped out of 18693
WR disabled & media.ffvpx.enabled set to false: 3819 dropped out of 18699
WR enabled & media.ffvpx.enabled set to false: 11821 dropped out of 18587

Comment 12

•

4 years ago

(In reply to Alastor Wu [:alwu] from comment #5)

Hi, when you clicked Publish, did you click Include hidden threads? Because I can't see WebContent process and its related threads on which the major media pipeline runs.

However, the weird thing is that I indeed saw MediaDecoderStateMachine thread in the list of Threads Filter in your report.

I renamed the Playback media threads as Controller, the UI wasn't updated to reflect that. So the media present no longer capture the playback thread

Comment 13

•

4 years ago

With such loss in performance when WR is enabled, I'm going to reassign that bug there.

Component: Audio/Video: Playback → Graphics: WebRender

Reporter

Comment 14

•

4 years ago

Jean-Yves, can we split off a new bug into the playback issue? The WebRender thing can be investigated separately, but as far as I can tell, playback is worse than Edge as it is.

Comment 15

•

4 years ago

(In reply to Asif Youssuff from comment #14)

Jean-Yves, can we split off a new bug into the playback issue? The WebRender thing can be investigated separately, but as far as I can tell, playback is worse than Edge as it is.

63% of frame drop vs 20% is likely the only thing that matter.

We are already aware across multiple bugs that Edge performs much better than we do when it comes to playback. They are doing zero-copy decoding which are are currently unable to achieve (this could be achieved with bug 1539735).
They also use a DXVA decoder directly while we are using a WMF media transform which adds a lot of unnecessary overhead.

Having said that, here your machine is clearly incapable of doing properly such content.

Now regarding WebRender, it could be that we are just reporting the frame drops differently when using webrender; :mstange has made some changes recently in regards to how frames dropped are reported in webrender; in which case it's just a red-herring that webrender is looking so bad here.

:mstange, what do you think?

Flags: needinfo?(mstange.moz)

Comment 16

•

4 years ago

(In reply to Asif Youssuff from comment #11)

Did some additional testing:

WR disabled: 4507 dropped out of 18693
WR disabled & media.ffvpx.enabled set to false: 3819 dropped out of 18699
WR enabled & media.ffvpx.enabled set to false: 11821 dropped out of 18587

Could you collect a profile in each of these configurations so that we can compare?

Flags: needinfo?(mstange.moz) → needinfo?(yoasif)

Comment 17

•

4 years ago

(In reply to Jean-Yves Avenard [:jya] from comment #15)

We are already aware across multiple bugs that Edge performs much better than we do when it comes to playback. They are doing zero-copy decoding which are are currently unable to achieve (this could be achieved with bug 1539735).
They also use a DXVA decoder directly while we are using a WMF media transform which adds a lot of unnecessary overhead.

This makes it sound like we're resigned to being worse. I'd like to be clear on the fact that, whenever we're worse, that's a valid bug.
Thank you Asif for filing this bug.

Comment 18

•

4 years ago

(In reply to Jean-Yves Avenard [:jya] from comment #15)

Now regarding WebRender, it could be that we are just reporting the frame drops differently when using webrender; :mstange has made some changes recently in regards to how frames dropped are reported in webrender; in which case it's just a red-herring that webrender is looking so bad here.

There should not be a difference in the reporting of dropped frames between WR and non-WR.

The video frame drops seem to all be occurring in the video sink. See the DiscardVideo markers on the MediaDecoderStateMachine thread:
https://share.firefox.dev/30QRDNK

Kris Taeleman (:ktaeleman)

Reporter

Comment 19

•

4 years ago

•

Edited

Markus,

Here is what I came up with:

WR enabled, media.ffvpx.enabled = true (default): https://share.firefox.dev/2PMpHED
WR enabled, media.ffvpx.enabled = false: https://share.firefox.dev/2DERQex
WR disabled, media.ffvpx.enabled = true: https://share.firefox.dev/31Gnccw
WR disabled, media.ffvpx.enabled = false: https://share.firefox.dev/3kBbzMC

Flags: needinfo?(yoasif)

Comment 20

•

4 years ago

@Sotaro: Could you take a look at what's going on here?

Blocks: gfx-82

Severity: -- → S2

Flags: needinfo?(sotaro.ikeda.g)

Priority: -- → P3

Andrew Osmond [:aosmond] (he/him)

Comment 21

•

4 years ago

WebRender+native compositor uses more GPU power. That might increase the frame drop. If the vp9 video was decoded by GPU, Bug 1460499 might reduce GPU update and might reduce frame drop.

Comment 22

•

4 years ago

Now that bug 1640526 has landed, can you still reproduce the perf being significantly worse with WebRender in the latest nightly? Thanks!

Flags: needinfo?(yoasif)

Comment 23

•

4 years ago

(In reply to Asif Youssuff from comment #19)

Markus,

Here is what I came up with:

WR enabled, media.ffvpx.enabled = true (default): https://share.firefox.dev/2PMpHED
WR enabled, media.ffvpx.enabled = false: https://share.firefox.dev/2DERQex
WR disabled, media.ffvpx.enabled = true: https://share.firefox.dev/31Gnccw
WR disabled, media.ffvpx.enabled = false: https://share.firefox.dev/3kBbzMC

Thanks! As far as I can tell, the difference between the WR on and off profiles are firmly on the decoding side.
WR on: https://share.firefox.dev/3hldq5n
WR off: https://share.firefox.dev/33hapxR
(both with media.ffvpx.enabled = true)

The WR on profile has a lot more DiscardVideo markers. We must be falling behind during decoding. This could either be due to extra GPU load (what Sotaro said), or maybe we're trying to decode for the wrong target timestamp. I don't know enough about how we determine the decoding target timestamps to say why WR would be making a difference here.

Kris Taeleman (:ktaeleman)

Reporter

Comment 24

•

4 years ago

(In reply to Andrew Osmond [:aosmond] from comment #22)

Now that bug 1640526 has landed, can you still reproduce the perf being significantly worse with WebRender in the latest nightly? Thanks!

Did another test.

WR enabled 12663 dropped frames out of 18386 (~69%)
WR disabed 4875 dropped frames out of 18562 (~26%)

Hope this helps.

Flags: needinfo?(yoasif)

Comment 25

•

4 years ago

@Sotaro: Do you have an idea what we could do here?

Comment 26

•

4 years ago

•

Edited

(In reply to Asif Youssuff from comment #19)

WR enabled, media.ffvpx.enabled = true (default): https://share.firefox.dev/2PMpHED
WR enabled, media.ffvpx.enabled = false: https://share.firefox.dev/2DERQex
WR disabled, media.ffvpx.enabled = true: https://share.firefox.dev/31Gnccw
WR disabled, media.ffvpx.enabled = false: https://share.firefox.dev/3kBbzMC

It seems that pref "media.ffvpx.enabled" did not affect to the profile results, since all of them have the following function. It seems that hardware decoder was used for vp9 video decoding.

WMFVideoMFTManager::Output()
WMFVideoMFTManager::CreateD3DVideoFrame()

Under MFTDecoder::Output() call, CDevice::WaitForSynchronizationObjectFromCpuCB() wait happened for very long duration. And the wait became longer when WebRender is enabled. From it, WebRender uses more GPU than non-WebRender. And WebRender caused the WaitForSynchronizationObjectFromCpuCB() wait longer. Then decoding speed seemed to become slower.

If we want to test software vp9 decoding, we also need to set pref "media.wmf.vp9.enabled=false" in addition to pref "media.ffvpx.enabled".

Comment 27

•

4 years ago

Asif, can you test software decode VP9 performance with pref "media.wmf.vp9.enabled=false"?

Flags: needinfo?(yoasif)

Reporter

Comment 28

•

4 years ago

Sotaro, I should have realized this even before trying it, but the performance is abysmal. Basically (literally) 100% dropped frames. This is a very slow CPU, so software decode at this resolution is impossible.

Flags: needinfo?(yoasif)

Kris Taeleman (:ktaeleman)

Updated

•

4 years ago

Summary: Stuttery playback of 1440p video on YouTube → Stuttery playback of 1440p video on YouTube with UHD 600

Comment 29

•

4 years ago

@Nical: Can you reproduce this on the cherryview intel ?

Flags: needinfo?(nical.bugzilla)

Comment 30

•

4 years ago

(In reply to Asif Youssuff from comment #28)

Sotaro, I should have realized this even before trying it, but the performance is abysmal. Basically (literally) 100% dropped frames. This is a very slow CPU, so software decode at this resolution is impossible.

Thank you for checking! From it, hardware decoding is necessary and gpu task needs to be reduced drastically. bug 1539735 could be one solution.

Flags: needinfo?(sotaro.ikeda.g)

Nicolas Silva [:nical]

Comment 31

•

4 years ago

@Nical: Can you reproduce this on the cherryview intel ?

Yeah, with that device, things work well in 720p but in 1440p playback gets stuck on a particular frame for a long time even though the timeline in the video control advances.

Flags: needinfo?(nical.bugzilla)

Jim Mathies [:jimm]

Updated

•

4 years ago

No longer blocks: gfx-82

Comment 32

•

3 years ago

Is there any news on this issue? Video playback performance in FF is terrible. Almost all the frames are dropped. I have to use Chrome to watch HQ videos on YT!

Bob Hood [:bhood]

Comment 33

•

2 years ago

Nical, can you please retest this and see if it remains an issue?

Flags: needinfo?(nical.bugzilla)

Nicolas Silva [:nical]

Comment 34

•

2 years ago

I don't have the cherryview low end device that could reproduce this anymore. I don't know of any work that would have moved the needle in that area.

Flags: needinfo?(nical.bugzilla)

Comment 35

•

2 years ago

For me it's reproducible on this video https://www.youtube.com/watch?v=7MIciimLLdM (or any other video on that channel) but not in 1440p. 1440p is ok but 1080p x2 speed is a slideshow. In Chrome 1080p x2 and even 1440p x2 playbacks are ok.

Intel UHD Graphics 617 1536 MB

Jim Mathies [:jimm]

Comment 36

•

2 years ago

•

Edited

(In reply to ris58h from comment #35)

For me it's reproducible on this video https://www.youtube.com/watch?v=7MIciimLLdM (or any other video on that channel) but not in 1440p. 1440p is ok but 1080p x2 speed is a slideshow. In Chrome 1080p x2 and even 1440p x2 playbacks are ok.

Intel UHD Graphics 617 1536 MB

This plays fine for me at 1x or 2x at 1440 and 1080. Intel(R) UHD Graphics 620.

Could you help us out by capturing and uploading a Firefox performance profile when this happens? Visit https://profiler.firefox.com/ for more information. When capturing, please select the Graphics profile in the drop down.

Jim Mathies [:jimm]

Updated

•

2 years ago

Flags: needinfo?(ris58h)

Comment 37

•

2 years ago

Could you help us out by capturing and uploading a Firefox performance profile when this happens? Visit https://profiler.firefox.com/ for more information. When capturing, please select the Graphics profile in the drop down.

I've recorded some Graphics profiles:

1080p x2. It hangs sometimes. https://share.firefox.dev/3TSUSgA
1080p x2 but I've entered and exited fullscreen mode. It hangs sometimes too. https://share.firefox.dev/3st8QKu
1440p x2 - just a static image but sound is playing. https://share.firefox.dev/3sqAqb3

Flags: needinfo?(ris58h)

Bob Hood [:bhood]

Comment 38

•

1 year ago

Sotaro, can you provide any insights on this?

Flags: needinfo?(sotaro.ikeda.g)

Comment 39

•

1 year ago

Original bug was for Windows PC. See comment 1.

Profile data from ris58h says that it is for MacOS. See Comment 37.

Window and MacOS uses different hardware decoder. Then they are different problems.

Profile data of Comment 37 does not have information of decoding side. If we get profile by using Settings "Media", the profile data seems to have more information.

Flags: needinfo?(sotaro.ikeda.g)

Comment 40

•

1 year ago

ris58h, can you get Firefox Profiler with Settings "Media"? Thank you.

Flags: needinfo?(ris58h)

Comment 41

•

1 year ago

:mstange, can you comment to profile data of Comment 37?

Flags: needinfo?(mstange.moz)

Comment 42

•

1 year ago

In the first two profiles, I see some "SkippedComposite" markers on the compositor thread with the reason "Too many pending frames", i.e. the Renderer thread wasn't able to keep up with the requested composites. I think the Renderer thread is CPU starved - there are long delays between the Compositor thread requesting a composite and the Renderer thread starting to work on a composite.
First profile with a view which shows the relevant markers: https://share.firefox.dev/40ix1e0
Second profile with the same view: https://share.firefox.dev/3HmyHKR
It might be a good idea to increase the priority of the Renderer thread on macOS, too.

In the third profile, the compositor never gets any decoded frames. There should be increasing frameIDs in the UpdateCompositedFrame markers, but there's just a single image: https://share.firefox.dev/3WSTQlL

Flags: needinfo?(mstange.moz)

Comment 43

•

1 year ago

It seems like it's better now. There are some lags but it's not a slideshow.
https://profiler.firefox.com/from-browser/calltree/?globalTrackOrder=0wg&hiddenGlobalTracks=1wd&hiddenLocalTracksByPid=804-0w9~3424-01~1258-0w2~9911-0w2~9247-0w2~11009-0w2~11005-0w2~6427-0w2~11098-0w2~2534-0w2~6432-0w2~2551-0w2~2552-0w2~10463-0w3~3466-02~3423-0wy8yawydyfwynyqyr~9279-0wfiqwxc&thread=Aj&v=8

Video https://www.youtube.com/watch?v=LXb3EKWsInQ 1440@60 HDR x2 in Theater Mode

Display resolution is scaled to larger text (1280x800)

MacBook Air
Retina, 13-inch, 2018
1,6 GHz Dual-Core Intel Core i5
Intel UHD Graphics 617 1536 MB
8 GB 2133 MHz LPDDR3

Flags: needinfo?(ris58h)

Comment 44

•

1 year ago

You need to upload the profile before you can share a link to it. Use the "Upload Local Profile" button in the top right corner.

Comment 45

•

1 year ago

(In reply to Markus Stange [:mstange] from comment #44)

You need to upload the profile before you can share a link to it. Use the "Upload Local Profile" button in the top right corner.

Sorry for a wrong link. https://share.firefox.dev/3Y6oXeE

Comment 46

•

1 year ago

:bradwerth, can you comment to comment 37 and comment 45?

Flags: needinfo?(bwerth)

Assignee

Comment 47

•

1 year ago

I don't have any additional insight into the profiles that have been uploaded so far.

I can sort-of reproduce this with a MacBook Pro with Intel UHD Graphics 630 running macOS 10.15, when pushing the quality up to 2160p. In that case I get some micro-stuttering. 1440p plays back fine. I've uploaded a profile of 5 seconds of playback. In the profile, I've tried to highlight only the relevant tracks, including both the media decoding and the compositor and rendering tracks.

I don't see anything actionable here. The only strange thing I noted is that the MediaPDecoder tracks spend most of their time in safeShift10BitBy6. That's fine, as long as the function is being inlined as intended. I assume the profiler is magically displaying the inlined stack, but if it's not, and the function has failed to be inlined, then that's something to fix.

Flags: needinfo?(bwerth)

Comment 48

•

1 year ago

(In reply to Brad Werth [:bradwerth] from comment #47)

I don't have any additional insight into the profiles that have been uploaded so far.

I can sort-of reproduce this with a MacBook Pro with Intel UHD Graphics 630 running macOS 10.15, when pushing the quality up to 2160p. In that case I get some micro-stuttering. 1440p plays back fine. I've uploaded a profile of 5 seconds of playback. In the profile, I've tried to highlight only the relevant tracks, including both the media decoding and the compositor and rendering tracks.

What video codec was being used in this profile?

Flags: needinfo?(bwerth)

Assignee

Comment 49

•

1 year ago

(In reply to Jeff Muizelaar [:jrmuizel] from comment #48)

What video codec was being used in this profile?

It's VP9. Stats for nerds shows it as vp09.02.51.10.01.09.16.09.00 (337) / opus (251).

Flags: needinfo?(bwerth)

Comment 50

•

1 year ago

I'm surprised it's not decoding in hardware.

Assignee

Comment 51

•

1 year ago

I'm going to split off the macOS playback issue into a new bug, leaving this for the originally-reported Windows playback issue. The platforms are different enough that we need to use different approaches for optimization on each, and we have different paths to achieve hardware decode, etc.

Summary: Stuttery playback of 1440p video on YouTube with UHD 600 → Stuttery playback of 1440p video on YouTube with UHD 600 on Windows

Assignee

Comment 52

•

1 year ago

I've opened a new Bug 1814535 for the macOS-specific playback issue. ris58h, if the problem is still happening for you on your Intel UHD Graphics 617 mac, it would be helpful if you would upload a profile recorded with "Media" presets and add a link to it in that Bug. Since your earlier profiles were recorded with macOS 13, you should be getting hardware decode and it would be useful to see how hardware decode can cause stuttering.

Flags: needinfo?(ris58h)

Updated

•

1 year ago

Flags: needinfo?(ris58h)

Comment 53

•

1 year ago

Asif, can you still reproduce this?

Flags: needinfo?(yoasif)

Assignee

Updated

•

1 year ago

Assignee: nobody → bwerth

Reporter

Comment 54

•

1 year ago

Jeff, I am unfortunately still seeing dropped frames. Here's a new profile: https://share.firefox.dev/3Lj5rbQ

Flags: needinfo?(yoasif)

Assignee

Comment 55

•

1 year ago

(In reply to Asif Youssuff from comment #54)

Jeff, I am unfortunately still seeing dropped frames. Here's a new profile: https://share.firefox.dev/3Lj5rbQ

I can't tell from that profile if there is a graphics issue, because it doesn't include the sampling from the Renderer thread. All I've discovered so far is that there's a big JavaScript garbage collection right before the first stutter. Asif, would you please record and post a profile using the "Graphics" Settings preset?

Flags: needinfo?(yoasif)

Comment 56

•

1 year ago

The profile has Renderer thread. It is hidden in GPU process. It could be enabled by right click on GPU process.

Assignee

Comment 57

•

1 year ago

(In reply to Sotaro Ikeda [:sotaro] from comment #56)

The profile has Renderer thread. It is hidden in GPU process. It could be enabled by right click on GPU process.

I was expecting to see one in the parent process. At least, I see a Renderer, CanvasRender, and Compositor thread in the parent process when I record with the "Graphics" setting.

Assignee

Comment 58

•

7 months ago

I'll try to reproduce on my slightly-newer Ice Lake system, Intel Iris Plus Graphics.

Flags: needinfo?(yoasif)

Bob Hood [:bhood]

Comment 59

•

7 months ago

Any luck with reproduction, Brad?

Flags: needinfo?(bwerth)

Assignee

Comment 60

•

6 months ago

(In reply to Brad Werth [:bradwerth] from comment #58)

I'll try to reproduce on my slightly-newer Ice Lake system, Intel Iris Plus Graphics.

On Ice Lake, it's very smooth. I'm going to mark this Bug as stalled until we have a profile that's showing something actionable. Again, all I can see in the last posted profile from comment 54 is that there's a JavaScript GC before the first frame stutter, but there's no activity on any thread during the stall period.

Severity: S2 → S3

Status: NEW → ASSIGNED

Flags: needinfo?(bwerth)

Keywords: stalled