1649645 - Constant high CPU with WebRender enabled and windows minimized

Status: RESOLVED DUPLICATE of bug 1580117

(Core :: Graphics: WebRender, defect, P3)

Description

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Attachment: about-support.txt

Over the last week or so, I have noticed constant high CPU in the parent process, and mozregression identified the recent changes to enable WebRender on macOS as the cause for me.

OS: macOS 10.14.6
Machine: MacBook Pro (15-inch, 2018) / MacBookPro15,1
GPUs: Intel UHD Graphics 630, Radeon Pro 560X

Normally, total Firefox CPU usage hovers around 60 - 80% for me, but when WebRender is enabled, it hovers around 250% CPU (around 3x).

WR on: https://share.firefox.dev/3dKX2cI
WR off: https://share.firefox.dev/2ZjdMCA

I'm happy to try anything you'd like to investigate this issue. If there's something I can do to help development wise, I'm happy to try to dig in and improve things, but I would need a decent amount of guidance.

Comment 1

[Jeff Muizelaar [:jrmuizel]]

How many windows did you have open during this?

Comment 2

[Jeff Muizelaar [:jrmuizel]]

So it looks like we're constantly painting during the WebRender profile. Do you have any guesses as to what the cause would be? (Is something animating?)

Comment 3

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #1)

How many windows did you have open during this?

Lots. My default profile has many windows, each with many tabs. about:support data claims 36 open windows, which sounds believable to me.

(In reply to Jeff Muizelaar [:jrmuizel] from comment #2)

So it looks like we're constantly painting during the WebRender profile. Do you have any guesses as to what the cause would be? (Is something animating?)

Hmm, no nothing comes to mind immediately... When I cycle through all of the non-minimized windows, they all appear to showing static, unchanging content. Are there any debugging tools I can use to pin down the source of the constant painting, like tracing it back to window or tab somehow?

Comment 4

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Using about:processes to look at all threads in the browser process, along with constant high CPU in the single Renderer thread, there are also around 20 WRRenderBackend threads constantly active (each using around 2 - 8% CPU) as well (along with even more that seem mostly idle). Would it be useful to have a new profile that also includes these backend threads as well?

I went through all of my open windows one by one (including minimized ones), and they all appeared to be static, with no visibly animating content, so I could not see any visual reason why they would be constantly painting.

I tried to manually hunt for an "expensive" window by closing one window, then checking the CPU usage. With WR on, the browser process with all windows open starts at around 250% CPU. For each window closed, CPU usage decreases by 2 - 10%, so it seems like WR is doing some amount of fixed, constant work in every window, even though the content is not changing visually.

As part of this window hunt, I happened to discover that if I manually close and reopen each window (Cmd-Shift-W, then Cmd-Shift-N) one at a time, then total browser process CPU usage after all of that window reopening hovers around 10%, which is great! This suggests to me that at Firefox startup, many WRRenderBackend threads become trapped spinning forever when they all launch simultaneously, but if instead they are slowly created (assuming there's one per window), then everything works out okay. What info should I collect to help investigate this further?

Comment 5

[Jeff Muizelaar [:jrmuizel]]

You can turn on gfx.webrender.debug.profiler=true. That should make it obvious which Windows are painting. If you see this again try that and let me know how many windows seem to be painting.

Comment 6

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Thanks for the tip, the profiler is quite a nice tool indeed! 😄

A few windows were showing Google Docs, which meant they were slowly painting frames each time the cursor blinks in the document. After choosing new tabs in those windows, every window reached a stable state (the frame counter in the profiler stopped increasing). Despite all windows at a stable state / not painting, the browser process was still using 250% CPU as mentioned above.

What else can I collect to try to highlight what the browser process might be spending time on when there's seemingly nothing to actually paint?

Comment 7

[Jeff Muizelaar [:jrmuizel]]

Can you collect a profile for the situation you described: i.e. no windows are visibly painting but cpu usage is still high?

Comment 8

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Sure, here's a fresh profile for the conditions of:

• WR on
• WR profiler UI disabled (in case it affects performance)
• No windows visibly painting
• 250% browser process CPU

Profile: https://share.firefox.dev/2ZPV0mI

I enabled profiling for the WRRenderBackend threads as well this time. The Renderer thread seems to be the most active (esp. given there should not be anything to paint), but some of the WRRenderBackend threads are also doing some work as well.

Comment 9

[Jeff Muizelaar [:jrmuizel]]

(In reply to J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow) from comment #8)

Sure, here's a fresh profile for the conditions of:

• WR on
• WR profiler UI disabled (in case it affects performance)
• No windows visibly painting
• 250% browser process CPU

Profile: https://share.firefox.dev/2ZPV0mI

I enabled profiling for the WRRenderBackend threads as well this time. The Renderer thread seems to be the most active (esp. given there should not be anything to paint), but some of the WRRenderBackend threads are also doing some work as well.

Can you get a new profile with the profiler UI on during the profile? The profile that you sent last time suggests that the profiler overlay should show painting happening. Can you also check if any of your windows are minimized?

Comment 10

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Here's a new profile with the profiler UI enabled:

• WR on
• WR profiler UI enabled
• No windows visibly painting
• 250% browser process CPU
• 35 total windows (10 windows in normal state across 2 window manager spaces, 25 minimized windows)

Profile: https://share.firefox.dev/395BdDX

It does indeed seem related to minimized windows! Once I restored all windows to normal state (so that no windows are minimised), CPU usage in the browser process dropped to almost nothing.

Comment 11

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

One possible difference from bug 1580117 is that I have not yet upgraded to 10.15 yet (though may do so soon). At the moment, I am using 10.14.6.

Comment 12

[Jeff Muizelaar [:jrmuizel]]

(In reply to J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow) from comment #11)

One possible difference from bug 1580117 is that I have not yet upgraded to 10.15 yet (though may do so soon). At the moment, I am using 10.14.6.

Can you try the steps in bug 1580117 and see if they reproduce for you? If they do can you try them with a fresh profile?

Comment 13

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #12)

Can you try the steps in bug 1580117 and see if they reproduce for you? If they do can you try them with a fresh profile?

Yes, that bug does seem to reproduce in a new profile. I added lots of notes in bug 1580117 comment 22.

When trying the basic.html test page from that bug in a new profile, I only see 20% constant CPU, instead of the 250% constant CPU that I see in this bug (when using my main profile for normal use). It's possible that bug 1580117 is indeed the root cause, and perhaps my main profile just has more complex pages, which take more time to generate frames and composite...?

Since it's not quite obvious to me yet why the CPU usage is so much higher, I would suggest we keep this bug open until we know more about what's happening here, as perhaps it's both bug 1580117 along with other confounding factors.

Comment 14

[Jessie [:jbonisteel] pls NI]

This would be a good bug to help investigate for Mac

Comment 15

[Miko Mynttinen]

(In reply to J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow) from comment #13)

Since it's not quite obvious to me yet why the CPU usage is so much higher, I would suggest we keep this bug open until we know more about what's happening here, as perhaps it's both bug 1580117 along with other confounding factors.

It seems that there has been progress with bug 1580117, do you still think that bug is separate from this one?

Comment 16

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

(In reply to Miko Mynttinen [:miko] from comment #15)

It seems that there has been progress with bug 1580117, do you still think that bug is separate from this one?

After some testing with the fixes from bug 1580117 applied, I am seeing around 30 - 50% browser process CPU usage with WR enabled, so I am now fairly confident they are the same issue. (It happens to be the case that bug 1580117 is a much worse perf issue with WR than the legacy stack, so it's all the more important that we get it resolved.)

I'll mark this as a duplicate and set bug 1580117 as blocking wr-mac-nightly directly, as that seems like the right thing to do (but of course please edit as needed).