1745996 - [WebRender Android] when Fenix comes to foreground, WebRender seems to grab ~50 MB RAM for each tab that gets viewed. Switching through many tabs leads to big growth in Fenix’s RAM, which only shrinks when Fenix goes to background.

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect)

Description

[Mark]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:100.0) Gecko/20100101 Firefox/100.0

Steps to reproduce:

I think this is a WebRender issue, if not I apologise & will find some other part of Fenix to blame :)

Fenix seems to grab an additional 50 MB of RAM for each tab which is brought into view. That extra RAM accumulates while Fenix is foregrounded and I switch between (already loaded) tabs. Just switching tabs, no reloading or scrolling. The extra RAM only seems to be released when Fenix is backgrounded. 50 MB per tab really mounts up, eg switching between just 10 tabs can ramp Fenix's total RAM to much more than 1 GB which is a problem for even a 4 GB or 6 GB phone. Chrom(ium) and I think Fenix with WebRender disabled do not show this RAM growth on static tab switching followed by RAM shrinkage on backgrounding.

Example:- Galaxy M32, 6GB RAM, killed off as many apps as possible for maximum RAM headroom. Today’s Fenix Nightly with 10 tabs open each with mozilla.org fully loaded. Fenix in background uses ~500 MB RAM, about the same as Chrome. Bring Fenix to the foreground. Still using ~500 MB. Switch through the 10 tabs (eg swipe toolbar) without scrolling or reloading. Fenix RAM use grows by about 50 MB each time the tab switches. After viewing all 10 tabs Fenix uses ~1000 MB RAM. RAM stays at this level until Fenix is backgrounded when it drops back to about 500 MB. Repeatable.

Chrom(ium) does not show this behavior under the same circumstances, its RAM use stays at ~500 MB whether it’s in the foreground or background or tabs are being switched.

Likewise I think that Fenix with WebRender disabled does not show this behavior, its RAM use stays constant ~500 MB.

Caveat: I don’t think I can disable WebRender on recent Nightly builds, so I had to go back to my old Xperia X Compact which is running Nightly 201215 17:01 from late 2020 so I could compare WebRender on/off. Things might have changed a lot since 2020 so perhaps I should not be pointing the finger at WebRender.

My hopelessly naive guess is that each time a tab becomes visible WebRender grabs a ~50 MB buffer, but then when the next tab becomes visible WebRender fails to release or recycle that buffer, it grabs a new one. The buffers don’t get released until Fenix goes into the background.

The practical result is that my little Xperia X Compact (2 GB RAM) will switch between about 30x mozilla.org tabs with WebRender off, but only about 15x mozilla.org tabs with WebRender on. When I exceed those limits it starts constantly reloading tabs as I swipe through them, and Android Device Monitor shows horrific memory pressure, almost no Free or Inactive RAM at all.

I’m using
Android Studio > Tools > Android > Android Device Monitor
Set it up to show Memory Usage and you have to click “Update from Device” frequently to see what's going on
Mostly monitoring Unknown RAM, the big blue pie slice, but also Free which shows much the same thing (because Inactive etc stay reasonably constant)

Maybe this is expected behavior but I think it has bad consequences for the Android memory manager. Android tends to keep its RAM full with cached apps and the memory manager keeps it full to a pressure threshold. If Fenix’s memory footprint grows/shrinks by many hundreds of MB in a few seconds it could cause the memory manager to start trimming and killing like crazy.

This might explain some of the anecdotes which say Fenix is terrible at handling large numbers of tabs compared to Chrom(ium) and Fennec. Certainly that’s what I see on my 2 GB Xperia X Compact (albeit it has an old build of Fenix). With WebRender OFF it can handle nearly twice the tabs (~30) that it can with WebRender ON (~15).

eg https://github.com/mozilla-mobile/fenix/issues/12731

Profile of Fenix in background, brought to foreground, swipe through 10 tabs, put back in background. Galaxy M32, 6 GB RAM, recent Nightly, Graphics presets

https://share.firefox.dev/30sQD5i

Actual results:

Fenix RAM grows as I switch between 10 static tabs, by many hundreds of MB. RAM shrinks back to where it was only when I background Fenix.

Expected results:

RAM should stay roughly constant when I'm just switching between tabs????????

Comment 1

[Mark]

Aha, about:memory says it's WebRender textures, I'm well pleased to have worked that out from scratch

Galaxy M32, 6 GB RAM, recent Nightly but I think it's the same on all version, 9x mozilla.org tabs open plus about:memory

bring Fenix to the foreground, gfx -> webrender -> textures using 235 MB
swipe through 10 tabs, gfx -> webrender -> textures using 784 MB
send Fenix to the background
bring Fenix to the foreground
gfx -> webrender -> textures using 251 MB

memory growth is almost exactly 50 MB per tab viewed.

Hope that helps, I think this is a big part of the "Fenix won't handle many tabs" problem in https://github.com/mozilla-mobile/fenix/issues/12731

Cheers & happy holidays

PS about:memory is very crashy, if I remember I will file something

Comment 2

[Mark]

Attachment: Screenshot_20211216-113920_Firefox Nightly.jpg

Comment 3

[Mark]

Attachment: Screenshot_20211216-113943_Firefox Nightly.jpg

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:jimm, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 5

[Jamie Nicol [:jnicol]]

Thanks for the detailed bug report Mark. The root cause of this is because Fenix uses a different "window" per tab, which currently means we use a separate webrender renderer instance per tab, each of which has its own texture cache as you describe. We have a long term plan to change that in bug 1654938.

Nical, am I right in thinking you made/planned to make a shorter term change to free memory for inactive renderers?

Comment 6

[Mark]

Hmm, interesting, thanks Jamie.

Is there anything hacky you can do in the short term? I think this is a major part of the frequent reloading https://github.com/mozilla-mobile/fenix/issues/12731 which does seem to be a big problem. I guess there might be other issues too, but perhaps they are obscured by WebRender's memory grab.

Is it still possible to revert to OpenGL rendering in current Nightly? If so I'd like to try that for a while, to see if there's any subjective reduction in reloading.

Comment 7

[Mark]

Is there anything hacky you can do in the short term?

For example "Minimize memory usage" in about:memory does the trick, it reduces gfx memory use by several hundred MB. It doesn't seem to have any obvious bad side effects.

Could you not just call "Minimize memory usage" every time the user switches tabs?

[ I'm sure that's a horrible hack. You can tell I am not a programmer ;) ]

Comment 8

[Nicolas Silva [:nical]]

(In reply to Jamie Nicol [:jnicol] from comment #5)

Nical, am I right in thinking you made/planned to make a shorter term change to free memory for inactive renderers?

There's https://phabricator.services.mozilla.com/D117918 but at the time I tried to land it it was causing some rather obscure failures. I'll have to give it another go eventually.

Could you not just call "Minimize memory usage" every time the user switches tabs?

What that patch does is actually not too far from that (it triggers the webrender specific part of minimize memory usage when a window (or android tab) has not been active for a certain amount of time.

Comment 9

[Mark]

(In reply to Nicolas Silva [:nical] from comment #8)

There's https://phabricator.services.mozilla.com/D117918 but at the time I tried to land it it was causing some rather obscure failures. I'll have to give it another go eventually.

I think it would be really useful to get this working, I think WebRender memory usage is one of several causes of https://github.com/mozilla-mobile/fenix/issues/12731 which is making users very very annoyed.

What that patch does is actually not too far from that (it triggers the webrender specific part of minimize memory usage when a window (or android tab) has not been active for a certain amount of time.

My guess is this would need to be fairly aggressive in freeing up memory. Fenix's memory footprint can balloon quite fast as users switch tabs, eg I can take Fenix's main process from ~500 MB to ~1300 MB in seconds just by switching through some tabs. I think this makes the memory trim & low memory killer daemons quite angry. Fenix's memory behavior is very different from Chrom(ium) whose memory footprint stays very stable except of course when many new tabs are being opened. I'm still trying to understand all this.

Side question:- does a WebRender renderer instance use much more memory than its OpenGL equivalent? In about:memory I see WR using eg 140 MB for a single page, that sounds like an awful lot on a low memory device, a 2 GB device might have <1 GB for user apps.

Thanks!

Comment 10

[Mark]

And also perhaps https://phabricator.services.mozilla.com/D117918 needs to connect into Fenix's onTrimMemory() handler so that when the Android memory manager starts sending onTrimMemory() messages to Fenix the excess memory used by WebRender gets released as well as pages being unloaded (or engines being suspended, I don't understand the difference)

My WILD guess is that part of the problem with https://github.com/mozilla-mobile/fenix/issues/12731 is that when Fenix gets onTrimMemory() messages it starts to free up relatively small amounts of RAM by unloading pages. Meanwhile WebRender is unnecessarily holding on to much larger amounts of RAM. So Fenix has to unload a lot more pages than it really needs to.

Comment 11

[[ex-Mozilla] Agi Sferro | :agi]

FYI I opened Bug 1750878 to respond to onTrimMemory in the main process, which should help with this when the device runs low on memory.

Comment 12

[[ex-Mozilla] Agi Sferro | :agi]

Attachment: Bug 1745996 - Send a memory pressure event when deactivating sessions.

WebRender retains about 50MBs of memory for every window. When switching
between lots of tabs on Android (where 1 tab = 1 window), this can cause
problems as the app will consume a significant amount of memory.

To avoid this problem, we send a memory pressure event whenever a session is
deactivated, which signals to WebRender that it should deallocate the memory.

This message is sent on a delay of 10s to avoid interfering with tab switching,
and we cancel the message if the tab becomes active again before we fire the
memory pressure event.

Co-Authored-By: Cathy Lu <[e-mail]>
Co-Authored-By: Jonathan Almeida [:jonalmeida] <[e-mail]>

Comment 13

[Mark]

Looks promising, I have some questions and concerns

(1) is the 10 second delay really necessary? My concern is I can switch through several tabs in 10 seconds, if each texture cache is 50-150 MB I could imagine WR holding on to ~400 MB of texture cache for several seconds. I think that could cause problems

• a low end phone really only has 600-700 MB of user available RAM, I think holding on to 100s of MB of cache could substantially degrade the overall UX.
• Fenix's memory footprint will still grow and shrink quite a lot quite quickly which I think is what makes the memory manager(s) angry even on a device with substantial memory. Angry memory manager(s) could lead to unnecessary trimming and killing, which leads to a poor UX.

(2) what would be the consequence of setting that timer to 0 seconds ie deallocating memory immediately on tab switch? I assume there would be some overhead to re-establish and perhaps some janky scrolling but fooling around with my phone it does not look awful and it would save 100s of MB

(3) what happens on Desktop? There is only 1 WR instance per window so when I switch tabs within the same window presumably the old texture cache is immediately deallocated/overwritten? That does not seem to be a problem, again is the 10 seconds really necessary?

To put it rudely, you WR guys seem very keen to use more RAM than absolutely necessary. Low end phones are incredibly limited on RAM, I am really keen to see WR use the absolute minimum RAM to give low end users the best experience. Every MB matters. I am concerned the 10 second delay might not solve this guy's problem, or many other users like him over on https://github.com/mozilla-mobile/fenix/issues/12731 .

To put the same thing another way, a WR texture cache is about the same size as an app or a page of web content. So for every texture cache which you hold onto, an app has to die.

Final question

(4) is this "deallocate on tab switch with 10 second delay" solution a short term kludge or a long term fix?

At the moment Fenix is more or less unusable on a low end device and it sucks with a few tabs even on a mid-ranger. Chrom(ium) is pretty good. I think you need to get as far away as possible from the suckage and as close to the Chrom(ium) ASAP. Fenix is pretty much unusable on a Galaxy S9 which is absurd, it needs to really really really good on an S9, ASAP. I think you need to free up every MB you possibly can, even if it slightly degrades the UX in other areas eg tab switching is janky.

Thanks all!

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/285565ecc581
Send a memory pressure event when deactivating sessions. r=jnicol,calu

Comment 15

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/285565ecc581