1682163 - Excessive memory use causing Firefox Quantum and other app crashes

Status: NEW

(Core :: Graphics: WebRender, defect, P3)

Description

[scrap]

Attachment: memory-report 20201213.json

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:83.0) Gecko/20100101 Firefox/83.0

Steps to reproduce:

1. Start Windows.
2. Fire up Firefox with an old session of ~30-80 tabs in ~5 windows mainly Youtube videos.
3. Browse the internet for 4 hours, mainly Youtube.

Actual results:

A. Firefox continues to use more and more memory until problems appear in the following order:

1. Sometimes: Errors in Youtube interface (e.g. the top-right profile menu shows blank, the add to playlist button becomes unresponsive etc.).
2. Later, blank (black) areas in Firefox - the tabs area, some websites areas.
3. The windows taskbar disappears.
4. File explorer has blank areas or erratic behavior, sometimes crashes.

B. Some resource heavy apps like GIMP or games will crash when launched.

Expected results:

Firefox should release the memory when needed by other apps.
Above all - Firefox should limit its memory use to SANE levels by whatever means necessary. Unload tabs from memory, drop video streams buffering etc.

WHAT I HAVE TRIED:

1. Clean profiles, clean installs, clean system installs, no addons.

Comments:

Addons:
I've experienced this problem across two separate installations of Windows 10 - one with 12 and one with 16 GB of RAM. If I remember correctly the similar behavior was present on that 16 GB RAM machine with Windows 7 x64.

I use Firefox manly with uBlock and sessions manager MySessions (with automatic session saving disabled - I ONLY use it to press the button and dump my session to bookmarks. For obvious reasons).

Other:
Always:
Strict tracking protection
pagefile limited to 4096 MB
latest drivers - vendor first

Comment 1

[scrap]

Attachment: memory-report 20201213 aft minim.json

Comment 2

[scrap]

Attachment: irfan_20201213_001908.jpg

Comment 3

[scrap]

Attachment: irfan_20201213_001934.jpg

Comment 4

[scrap]

Attachment: irfan_20201213_001956.jpg

Comment 5

[Peter Magyari (Desktop QA)]

Andrew, got another memory report for you, could you take a look over it and see if something stands out?

Thanks!

Comment 6

[Andrew McCreight [:mccr8]]

Nothing looks out of the ordinary in the memory report. Memory looks like it is being used for normal web page stuff, and not really video things, so maybe the video playback code is already dropping buffers as needed.

The limited pagefile might be causing the issues you are seeing. You could try increasing that or installing an addon to unload tabs (or "suspend tabs" as some of them call it). We have some work in progress to unload tabs that aren't being used, but it has been a lengthy project.

Comment 7

[Kim Moir [:kmoir] ET]

Hi scrap - were you able to try the suggestions in comment #6 re pagefile - did that make a difference?

Comment 8

[Markus Stange [:mstange]]

Blank areas in the UI might indicate that you're running out of GPU memory. Could you attach the contents of your about:support (or whatever subset you're comfortable sharing), and maybe other information about GPU memory use that you can get from the task manager?

Comment 9

[scrap]

(In reply to Kim Moir [:kmoir] ET from comment #7)

Hi scrap - were you able to try the suggestions in comment #6 re pagefile - did that make a difference?

Since some time now I've been running with pagefile "System managed". The issue persists. I've been using the [Auto Tab Discard] addon for a week now and surely it does its job! I hope you don't consider it anything else than a workaround.

Comment 10

[scrap]

(In reply to Markus Stange [:mstange] from comment #8)

Blank areas in the UI might indicate that you're running out of GPU memory. Could you attach the contents of your about:support (or whatever subset you're comfortable sharing), and maybe other information about GPU memory use that you can get from the task manager?

I haven't thought of that. On the other hand:

1. I constantly monitor task manager when using Firefox and during the "overuse" events it's clearly the system memory use which goes up to the cork.
2. I've been running games like ArmA 3 on med-high on this system.

That said I'm disabling the Auto Tab Discard right now to hunt for a juicy report.

Important:
My notebook is a dual graphics one. I only use the dedicated GPU for gaming. So the case is for my integrated Vega 10 GPU on Ryzen 2700U, for which 1 GB of dedicated GPU is reported, with 5.5 GB of potential shared GPU memory available. So far I've never seen it resort to shared memory.
HWinfo report attached.

Comment 11

[scrap]

Attachment: ideapad330sryzen2700uVega10plusRad540X.HTM

HWinfo detailed report

Comment 12

[scrap]

Attachment: firefox memory leak 2021-02-21.7z

A new memory report alongside shots of GPU memory state. The Explorer.exe error (on an attempt to copy a single file) is to me a clear indication the use of system memory is the culprit. Also, I haven't seen Firefox crash for quite some time now. It now gratiously consistently crashes other apps.

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+1]]

For the GPU process the memory reporter shows that most memory is used for the texture cache:

828.91 MB (100.0%) -- gfx
└──828.91 MB (100.0%) -- webrender
   ├──695.90 MB (83.95%) -- textures
   │  ├──575.29 MB (69.40%) ── texture-cache
   │  ├───84.28 MB (10.17%) ── swap-chains
   │  ├───30.75 MB (03.71%) ── depth-targets
   │  └────5.58 MB (00.67%) ++ (4 tiny)

Comment 14

[Markus Stange [:mstange]]

That's a lot of texture cache memory for only 5 windows. For comparison, I have 3 windows and 4MB of texture-cache memory.

What Firefox version are you using? If you use Nightly, is the situation improved?

Comment 15

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Nightly can actually be downloaded here: https://www.mozilla.org/en-US/firefox/channel/desktop/#nightly

Comment 16

[scrap]

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #13)

For the GPU process the memory reporter shows that most memory is used for the texture cache:

828.91 MB (100.0%) -- gfx
└──828.91 MB (100.0%) -- webrender
   ├──695.90 MB (83.95%) -- textures
   │  ├──575.29 MB (69.40%) ── texture-cache
   │  ├───84.28 MB (10.17%) ── swap-chains
   │  ├───30.75 MB (03.71%) ── depth-targets
   │  └────5.58 MB (00.67%) ++ (4 tiny)

I don't know whether Nightly behaves similarly. Be the judge yourself. Fresh clean Nightly 88. 6 freshly opened LinkedIn tabs, signle window:
212.78 MB (100.0%) -- gfx
└──212.78 MB (100.0%) -- webrender
├──195.17 MB (91.72%) -- textures
│ ├──120.72 MB (56.73%) ── texture-cache
│ ├───27.27 MB (12.81%) ── render-texture-hosts
│ ├───27.25 MB (12.81%) ── depth-targets
│ ├───15.20 MB (07.14%) ── swap-chains
│ ├────4.00 MB (01.88%) ── upload-staging-textures
│ └────0.73 MB (00.35%) ++ (4 tiny)

added 3 YT tabs, of which 1 is playing a generic visualisation music video:
245.11 MB (100.0%) -- gfx
└──245.11 MB (100.0%) -- webrender
├──223.04 MB (91.00%) -- textures
│ ├──139.07 MB (56.74%) ── texture-cache
│ ├───32.05 MB (13.08%) ── render-texture-hosts
│ ├───27.25 MB (11.12%) ── depth-targets
│ ├───15.24 MB (06.22%) ── swap-chains
│ ├────8.00 MB (03.26%) ── upload-staging-textures
│ └────1.42 MB (00.58%) ++ (4 tiny)
└───22.06 MB (09.00%) -- images/mapped_from_owner
├───7.92 MB (03.23%) ++ (7 tiny)
├───6.58 MB (02.68%) ++ pid=984
├───3.97 MB (01.62%) ++ pid=12028
└───3.59 MB (01.47%) -- pid=10584
├──3.52 MB (01.43%) ── image(1280x720, compositor_ref:1, creator_ref:1)/decoded-nonheap
└──0.08 MB (00.03%) ++ (4 tiny)

└───17.62 MB (08.28%) -- images/mapped_from_owner
├───8.66 MB (04.07%) ++ pid=984
├───3.97 MB (01.87%) ++ pid=12028
├───2.81 MB (01.32%) ++ (4 tiny)
└───2.18 MB (01.02%) ++ pid=13460

Comment 17

[scrap]

AMD Radeon driver (AMD-provided, not OEM-provided; yes, it's the way), 21.1 to 21.2 series. Driver always up-to-date at the time of writing/reporting, except for 21.2.3 update I'm currently postponing.
For reference
https://www.amd.com/en/support/kb/release-notes/rn-rad-win-21-2-3

Comment 18

[Jim Mathies [:jimm]]

Glen, curious if you can add insight here. Is the texture cache here expected based on the use scenario?

Comment 19

[Glenn Watson [:gw]]

The reported memory amounts do seem quite high, from first glance - typically we try to keep the texture cache to be < 100 MB GPU memory. Picture cache tiles can also add some to that, and depend on things such as screen resolution.

It also seems a lot higher in comment 13 than the most recent comments, so maybe it has been improved in nightly recently.

It's possible there is some reason for it to be this high (youtube does tend to load a lot of image preview thumbnails which are quite large), but it would need more investigation to be able to say whether this is expected or a bug.

Comment 20

[scrap]

(In reply to Glenn Watson [:gw] from comment #19)

The reported memory amounts do seem quite high, from first glance - typically we try to keep the texture cache to be < 100 MB GPU memory. Picture cache tiles can also add some to that, and depend on things such as screen resolution.

It also seems a lot higher in comment 13 than the most recent comments, so maybe it has been improved in nightly recently.

It's possible there is some reason for it to be this high (youtube does tend to load a lot of image preview thumbnails which are quite large), but it would need more investigation to be able to say whether this is expected or a bug.

Every time I post memory use info, I try to include the number of windows, tabs and type of content.

As for pushing through with this bug, may I add any other information to help with this?

Comment 21

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Scrap, maybe you could (temporary) install my PerfChaser extension to keep track of memory usage for Firefox processes. It might help us to figure out which website and activity could cause it. To do that you would have to make use of [Firefox Nightly] (https://www.mozilla.org/en-US/firefox/channel/desktop/#nightly) or Firefox DevEdition. The extension can be found at: https://github.com/whimboo/perfchaser

Comment 22

[scrap]

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #21)

Scrap, maybe you could (temporary) install my PerfChaser extension to keep track of memory usage for Firefox processes. It might help us to figure out which website and activity could cause it. To do that you would have to make use of [Firefox Nightly] (https://www.mozilla.org/en-US/firefox/channel/desktop/#nightly) or Firefox DevEdition. The extension can be found at: https://github.com/whimboo/perfchaser

Sorry for sounding like a noob, but I reviewed my notes and there's a clear warning - "nightly messes up with profiles of the regular installation!!!". This also explains why I haven't touched Nightly for a year now. Are you aware of any such problems?

In the meantime I tried a portableapps.com version and it misbehaves as seemingly the separate windows are not aware of the common profile. I have no idea what the hell is this.

That said a did manage to run perfchaser in one of the windows. With ~10 tabs it showed this see perfchaser 1 and perfchaser 2 attached png's.

Comment 23

[scrap]

Attachment: perfchaser 1.png

perfchaser

Comment 24

[scrap]

Attachment: perfchaser 2.png

Comment 25

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to scrap from comment #22)

Sorry for sounding like a noob, but I reviewed my notes and there's a clear warning - "nightly messes up with profiles of the regular installation!!!". This also explains why I haven't touched Nightly for a year now. Are you aware of any such problems?

There might be risks, yes. But so far I haven't had dataloss. FYI for testing purposes you could duplicate the profile folder on disk, and use that duplicate as the profile for Firefox Nightly for testing purposes. Otherwise the Developer Edition is mostly identical to the Firefox Beta releases. So maybe that would be a better option for you.

In the meantime I tried a portableapps.com version and it misbehaves as seemingly the separate windows are not aware of the common profile. I have no idea what the hell is this.

I'm not sure what you mean here. If you want to run a separate Firefox process with a different profile in parallel you would have to set the environment variable MOZ_NO_REMOTE=1 or pass -no-remote as command line argument.

That said a did manage to run perfchaser in one of the windows. With ~10 tabs it showed this see perfchaser 1 and perfchaser 2 attached png's.

Those values are not that dramatic. Also just snapshots aren't that helpful. You want to observe the memory load over time. Maybe there are cases when it grows for a particular process by hundreds of MB. Being able to replicate that would help to figure out the reproduction steps.

Comment 26

[scrap]

OK, I've just enabled sync and moved to Nightly entirely. Instead of copying my profile over from the stable version I installed my usual few addons (without the AutoTabDiscard of course) and just relied on Sync for Bookmarks and Passwords. I've disabled syncing of Options.

@:jimm @[e-mail] What does the new status mean? VVV
Blocks: gfx-stalled
No longer blocks: gfx-triage

Comment 27

[scrap]

Got something! While it's not what I was hunting for, i.e. a normal use case of an evening multi0window session with an excessive memory use, I've stumbled upon an outright memory leak! The attached archive contains all information I was physically able to gather. Timestamps on the reports and the screenshots allow for cross examination.

Comment 28

[scrap]

Attachment: nightly memory leak 2021-03-21.7z

includes the session backup

Comment 29

[scrap]

Well, there had to be something to it... Straight from "What's new" of Firefox 93.0.x
"When available system memory is critically low, Firefox on Windows will automatically unload tabs based on their last access time, memory usage, and other attributes. This should help reduce Firefox out-of-memory crashes. Switching to an unloaded tab automatically reloads it."