1714069 - Degraded performance after upgrading to Firefox 89 on Linux

Status: VERIFIED FIXED

(Core :: Graphics: WebRender, defect)

Description

[Steve C]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:89.0) Gecko/20100101 Firefox/89.0

Steps to reproduce:

FIrefox is using significantly more CPU after upgrading from Firefox 88 to 89. The increased CPU is noticeable on loading any page, typing into the address bar, and typing into a text box like this bug report.

Actual results:

As I type this, the main Firefox process is using over 100% of one CPU and the other Firefox processes are each using around 12%.

If I load https://www.ozbargain.com.au, the main process uses around 240% for several seconds, and the remaining processes around 25%.

Consequently my laptop is running hotter and the fan makes a lot of noise every time I load a new page.

Expected results:

Lower CPU. I will downgrade to Firefox 88 to try to get some comparative numbers.

Comment 1

[Steve C]

After downgrading to Firefox 88.0.1, typing into a text box peaks at around 23% of one core and typically sits at around 10%.

Loading https://www.ozbargain.com.au, the main process peaks at around 75% and falls back to ~5% after one second.

Obviously this isn't very scientific but there is clearly a significant difference.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Performance' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 3

[Sean Feng [:sefeng]]

Hi Steve, If you can reproduce this reliably, can you try mozregression to find which revision caused the regression?

Comment 4

[Steve C]

I have run mozregression, which showed the problem started prior to Firefox 88. This surprises my because I have no issues with my 88.0.1 install (Gentoo distribution, firefox-bin package).

But in any case, it does identify the problematic commit.

11:20.96 INFO: Last good revision: 1c59701a68c9e52fcdc0bf02c02eb1dca962bc6f
11:20.96 INFO: First bad revision: c2a14298ecf313c001958c37d49cb5d9e6b7894a
11:20.96 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=1c59701a68c9e52fcdc0bf02c02eb1dca962bc6f&tochange=c2a14298ecf313c001958c37d49cb5d9e6b7894a

Comment 5

[Jeff Muizelaar [:jrmuizel]]

Steve, can you post the graphics section of about:support?

Comment 6

[Jeff Muizelaar [:jrmuizel]]

Ideally from 88 and 89

Comment 7

[Steve C]

Attachment: about:support graphics section, Firefox 88.0.1

Comment 8

[Steve C]

Attachment: about:support graphics section, Firefox 89.0

Comment 9

[Robert Mader [:rmader]]

Interesting. So in both cases llvmpipe is used instead of a hardware driver - is that intended? Does you setup not have any hardware GPU? What changed is that we don't detect llvmpipe as a software driver any more, thus running HW-WR on llvmpipe instead of SW-WR - the later being much more optimized for running on the CPU.

Comment 10

[Robert Mader [:rmader]]

IIUC there're two places which we should fix:

• https://searchfox.org/mozilla-central/source/toolkit/xre/glxtest.cpp#530 - we only make sure to detect that a GPU is accelerated, not the opposite
• https://searchfox.org/mozilla-central/source/widget/gtk/GfxInfo.cpp#283 - we should already detect llvmpipe as a know software driver, but it apparently doesn't work for EGL

Comment 11

[Robert Mader [:rmader]]

I can reproduce this by using LIBGL_ALWAYS_SOFTWARE=1. Looks easy to fix.

Steve: you can work around this for the time being by setting gfx.webrender.software to true (or by fixing your setup in case you actually do have a GPU).

Comment 12

[Steve C]

I do have a GPU so clearly there is some issue with my setup. Setting gfx.webrender.software to true makes the performance go back to normal.

Thanks everyone!

Comment 13

[Robert Mader [:rmader]]

Attachment: Bug 1714069 - Block Mesa software drivers for HW-WR, r=aosmond

We currently allow software drivers such as llvmpipe to run HW-WR -
in nightly and apparently even in release.
Explicitly make these drivers always use SW-WR instead.

While on it, improve our detection for software drivers in glxtest.
This may become handy for new software drivers such as zink+lavapipe.

Comment 14

[Pascal Chevrel:pascalc]

Tracking for 89, I might take a safe patch in a dot release.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/22f7bdbfbd92
Block Mesa software drivers for HW-WR, r=aosmond

Comment 16

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/22f7bdbfbd92

Comment 17

[BugBot [:suhaib / :marco/ :calixte]]

Since the status are different for nightly and release, what's the status for beta?
For more information, please visit auto_nag documentation.

Comment 18

[Robert Mader [:rmader]]

Comment on attachment 9225062 [details]
Bug 1714069 - Block Mesa software drivers for HW-WR, r=aosmond

Beta/Release Uplift Approval Request

• User impact if declined: A small set of users will have massively reduced performance. These may be relatively common cases such as virtual machines.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: Start Firefox with LIBGL_ALWAYS_SOFTWARE=1 on Linux. Before this path, in some setups this will make us use "Webrender" instead of "Webrender (Software)". After this patch it should always use "Webrender (Software)" (or "Basic).
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): The worst thing that can happen is that some devices use SW-WR instead of HW-WR. That implies a little performance drop, but doesn't make the browser unusable.
• String changes made/needed:

Comment 19

[Robert Mader [:rmader]]

Comment on attachment 9225062 [details]
Bug 1714069 - Block Mesa software drivers for HW-WR, r=aosmond

Beta/Release Uplift Approval Request

• User impact if declined: A small set of users will have massively reduced performance. These may be relatively common cases such as virtual machines.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: Start Firefox with LIBGL_ALWAYS_SOFTWARE=1 on Linux. Before this path, in some setups this will make us use "Webrender" instead of "Webrender (Software)". After this patch it should always use "Webrender (Software)" (or "Basic).
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): The worst thing that can happen is that some devices use SW-WR instead of HW-WR. That implies a little performance drop, but doesn't make the browser unusable.
• String changes made/needed:

Comment 20

[Robert Mader [:rmader]]

@Pascal: the worst case scenario has actually happened and some devices now fall back to SW-WR that shouldn't. A patch is coming in bug 1714897.

Comment 21

[Raluca Popovici, Desktop QA]

I followed the steps from comment 19 and after update to Fx89 there is no reduced performance. @Steve C, could you please retest this on Nightly 91 to see if the issue is fixed for you?

Comment 22

[Steve C]

Since I raised this issue I have fixed my environment so that hardware acceleration is working properly, and therefore I can only reproduce it by setting LIBGL_ALWAYS_SOFTWARE=1. With that environment variable I can reproduce the issue with Firefox 89 but not with Nightly 91.

Comment 23

[Raluca Popovici, Desktop QA]

I will close this bug as verified fixed because Steve C confirms the fix on Nightly 91.

Comment 24

[Julien Cristau [:jcristau]]

I'll hold off on the beta uplift until the issues with amdgpu and nvidia are sorted.

Comment 25

[Julien Cristau [:jcristau]]

Comment on attachment 9225062 [details]
Bug 1714069 - Block Mesa software drivers for HW-WR, r=aosmond

approved for 90.0b7

Comment 26

[Julien Cristau [:jcristau]]

https://hg.mozilla.org/releases/mozilla-beta/rev/429878166686

Comment 27

[Catalin Sasca, Desktop Test Engineering [:csasca]]

Steve, can you please verify if the issue is fixed for you on Firefox 90 as well?

Comment 28

[Steve C]

Looks good, I cannot reproduce the original issue with FIrefox 90.0b7.

Comment 29

[Catalin Sasca, Desktop Test Engineering [:csasca]]

Thanks for looking into it, Steve!

Comment 30

[Julien Cristau [:jcristau]]

Bug 1715902 fixed this in 89.0.1.