1452995 - about:newtab memory leak

Status: RESOLVED DUPLICATE of bug 1436615

(Firefox :: New Tab Page, defect, P1)

Description

[Matthew Kidd]

Attachment: about:newtab memory after minimize memory use

about:newtab seems to be leaking memory via strings in js-zone which appear to contain images. Take a look at the two attached partial screenshots of about:memory output. The first is after clicking minimize memory use in the about:memory page. At this point Firefox is consuming about 3.5 GB across all processes (yes, I have a lot of tabs). The second screenshot shows the about:memory use about 10 hours later when total memory use is 5.4 GB.

Of some note is that about:newtab is consuming near 150 MB and when one of these is expanded in the tree, I see several strings of about 10 MB each which appear to be storing PNG images. That a lot of memory for a tab that isn't yet showing me anything I really care about. FWIW, I have the default new tab preferences except that I have unchecked "Recommended by Pocket".

I do appreciate the performance of FFQ but the benefits are negated when my box starts to swap. Yes, it's an older box but it's still a quad-core with 8 GB of RAM. Before FFQ, this set of tabs would startup with consuming about 1.2 GB on a pre-FFQ 32-bit version of Firefox and maybe work its way up to 2.5 GB across all 32-bit processes. Now it's just a total memory hog. I don't think the issue is the number of tabs per se because many of them are not clicked on during a session and not that many tabs show up in the about:memory report if I search for "top(". So I'm looking for wasted memory and the about:newtab memory consumption seems to be one issue but I suspect there are more.

Comment 1

[Matthew Kidd]

Attachment: about:newtab memory use after 10 hour of "idling"

Comment 2

[Matthew Kidd]

I can't see the forest from the trees in the about:memory reports so I wrote a Perl program to digest the JSON output and tally up memory explicitly allocated to each tab and also memory explicitly allocated but not to a specific tab. For the case mentioned above I have the following:

[before GC] Total tab memory: 1768.05 MB  about:newtab total: 1357.40 MB  other: 1576.22 MB
[after  GC] Total tab memory:  792.45 MB  about:newtab total:  352.71 MB  other: 1036.34 MB

Before garbage collection (after the 10 hour idle), most of the 1.7 GB explicitly allocated to tabs is being used by about:newtab tabs. After garbage collection, the about:newtab memory use falls by 1.0 GB! A close look at other tabs shows that GC has essentially no impact on their memory use, either in aggregate as can be inferred from the numbers above, or individually. Note: GC did reduce the use of memory not assigned to a specific tab by about 500 MB.

Moreover, I see a lot of distinct about:memory tabs, where I assume a different id values in 'path' field of JSON memory report output distinguishes such. Here is sample of the program output showing this (memory is total with js-zone (JSZ), js-compartment (JSC), and DOM use totals also shown):

Sorted by total memory use
233.86  JSZ:216.00 JSC:  7.88 DOM: 8.53  about:newtab
233.81  JSZ:215.95 JSC:  7.87 DOM: 8.53  about:newtab
233.78  JSZ:215.95 JSC:  7.86 DOM: 8.48  about:newtab
233.78  JSZ:215.96 JSC:  7.82 DOM: 8.53  about:newtab
 62.38  JSZ: 50.57 JSC:  5.12 DOM: 5.23  about:newtab
 62.35  JSZ: 50.57 JSC:  5.06 DOM: 5.22  about:newtab
 62.34  JSZ: 50.57 JSC:  5.09 DOM: 5.23  about:newtab
 48.37  JSZ: 38.50 JSC:  7.17 DOM: 0.48  https:\\blog.frizk.net\2018\03\total-meltdown.html
 41.08  JSZ: 30.96 JSC:  3.90 DOM: 4.81  about:newtab
 38.40  JSZ: 28.20 JSC:  3.90 DOM: 4.80  about:newtab
 38.36  JSZ: 28.19 JSC:  3.93 DOM: 4.80  about:newtab
 38.35  JSZ: 28.19 JSC:  3.90 DOM: 4.78  about:newtab
 28.54  JSZ: 20.03 JSC:  2.35 DOM: 4.67  about:newtab
 25.20  JSZ: 16.84 JSC:  2.35 DOM: 4.57  about:newtab
 25.17  JSZ: 16.82 JSC:  2.35 DOM: 4.58  about:newtab
 22.13  JSZ:  8.36 JSC:  8.27 DOM: 1.58  https:\\photos.google.com\share\AF1QipMEaZayFyyIm4e...

I don't have a bunch of about:newtab tab open. I'm guessing these are in the tab history after opening a new tab and navigating somewhere. I think we would be better off flushing memory used by about:newtab as soon as the user goes somewhere else, regenerating the page if the user navigate backwards.

This is a serious issue. Runaway memory use makes the product look bad and we can't exactly expect the average user to go to about:memory and regularly click the "Minimize memory usage" button.

Comment 3

[Matthew Kidd]

Let's see if we can make a lighter about:newtab. From the New Tab Preferences, I unchecked Top Sites, Recommended by Pocket (which I had already unchecked), Highlights, and Snippets, and then restarted the browser. I took a memory snapshot shortly thereafter. I still have memory explicitly allocated to 15(!) about:newtab entries (distinct ids). Where do they come from?! But at least each one is now only consuming about 4.2 MB (2.0 js-zone, 1.44 js-compartment, 0.07 DOM) which is a definite improvement.

Comment 4

[Matthew Kidd]

Poking around with Firefox 59 on other boxes and/or other profiles, it appears there is one about:newtab entry per window in the memory report, except for private browsing windows which have a very lightweight 0.72 MB about:privatebrowsing entry. Do we need this overhead for a tab that isn't even visible? Is it an optimized "always ready" sort of thing? Do we have one per window because each window may have a different width and height?

I took a memory snapshot just now after 15 hours of idle time after restarting Firefox with the light about:newtab settings. There is still a memory leak or other memory waster, though it is not as bad as seen before. Here is the output of my Perl program:

Sorted by total memory use (MB)
 53.06  JSZ: 50.56 JSC:  1.75 DOM: 0.07  about:newtab
 53.04  JSZ: 50.56 JSC:  1.75 DOM: 0.07  about:newtab
 53.03  JSZ: 50.56 JSC:  1.75 DOM: 0.07  about:newtab
 26.09  JSZ: 23.60 JSC:  1.74 DOM: 0.07  about:newtab
 26.06  JSZ: 23.60 JSC:  1.74 DOM: 0.07  about:newtab
 26.06  JSZ: 23.60 JSC:  1.74 DOM: 0.07  about:newtab
 26.06  JSZ: 23.60 JSC:  1.74 DOM: 0.07  about:newtab
  4.87  JSZ:  2.76 JSC:  1.37 DOM: 0.07  about:newtab
  4.85  JSZ:  2.76 JSC:  1.37 DOM: 0.07  about:newtab
  4.84  JSZ:  2.75 JSC:  1.37 DOM: 0.07  about:newtab
  4.38  JSZ:  2.34 JSC:  1.30 DOM: 0.07  about:newtab
  3.84  JSZ:  1.81 JSC:  1.30 DOM: 0.07  about:newtab
  3.82  JSZ:  1.80 JSC:  1.29 DOM: 0.07  about:newtab
  3.82  JSZ:  1.80 JSC:  1.29 DOM: 0.07  about:newtab

450 MB wasted on about:newtab windows that aren't even being displayed!

Again the problem seems due to the store of PNG images in strings, e.g. here are some large allocations:

1.13 explicit/window-objects/top(about:newtab, id=8589934605)/js-zone(0x10afe000)/strings/string(length=11922, copies=96, "data:image\png;base64, ..." )/malloc-heap/latin1
3.75 explicit/window-objects/top(about:newtab, id=8589934605)/js-zone(0x10afe000)/strings/string(length=37962, copies=96, "data:image\png;base64, ..." )/malloc-heap/latin1
4.50 explicit/window-objects/top(about:newtab, id=8589934605)/js-zone(0x10afe000)/strings/string(length=47358, copies=96, "data:image\png;base64, ..." )/malloc-heap/latin1

Values at the left are in MB and I replaced (truncated) with ellipses in my code that prints out allocations only above a specified size.

I am not at all clear about what is meant by copies=## in these path strings from the Firefox JSON memory reports. On the surface it looks like some sort of reference counting but given the amount of memory being consumed by the entire Firefox process it seems like there really are 96 copies of these of these strings rather than 96 references to each of three strings.