1404759 - Firefox memory leak/overly high allocation per tab for some pages/sites

Status: RESOLVED WORKSFORME

(Core :: General, defect)

Description

[u285848]

Attachment: Screen Shot 2017-10-01 at 8.06.14 PM.png

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:57.0) Gecko/20100101 Firefox/57.0
Build ID: 20170928180207

Steps to reproduce:

Open the following pages in latest Firefox and leave them open:
https://laughingspatula.com/chicken-avocado-burger/
https://www.washingtonpost.com/news/the-switch/wp/2017/09/29/elon-musk-says-his-next-spaceship-could-not-only-take-to-you-the-moon-and-mars-but-from-n-y-to-london-in-29-minutes/?utm_term=.7a30c6e35c9d
https://news.google.com/news/?ned=us&hl=en
https://mail.google.com/ (open to a gmail account with a lot of email and contacts if it matters)


Actual results:

After some hours, memory is up to 3-4GB from one tab. macOS 10.12.6. FF 57.0b3 64-bit. Reproducible. All plugins/extensions are disabled.


Expected results:

Firefox shouldn't used as much memory.

Comment 1

[Eric Rahm [:erahm]]

Thanks for the report. Can you reproduce this locally and attach an about:memory report [1]? Can you also test out in safe mode [2]? It's possible some local settings may be causing issues.

[1] https://developer.mozilla.org/en-US/docs/Mozilla/Performance/about:memory
[2] https://support.mozilla.org/en-US/kb/troubleshoot-firefox-issues-using-safe-mode

Comment 2

[u285848]

Attachment: memory-report.json.gz

Comment 3

[u285848]

Attachment: firefox - memory.txt

Comment 4

[u285848]

Attachment: Screen Shot 2017-10-23 at 7.21.16 AM.png

Comment 5

[u285848]

For the logs I just posted from this morning, I was using (macOS) FF 57.0b8 (64-bit) and the memory allocation was high and was spinning out-of-control. It was up to 4.99GB by the time I stopped Firefox. I leave Firefox up most of the time, so I'm not sure how long it had been running.

Comment 6

[Andrew McCreight [:mccr8]]

The memory usage in the about:memory log looks fine. The vsize is really big, but that number is not very accurate on OSX, is my understanding. The description says "This figure is of limited use on Mac, where processes share huge amounts of memory with one another." Maybe the fact that most of it is compressed means it isn't really using physical RAM? I'm not sure what that stat means. Maybe we're churning through memory (pages with ads can end up allocating and freeing a lot of memory due to reloading), but free it when we're done (and MADVISE it) but the OS is still counting it against us?

Comment 7

[Eric Rahm [:erahm]]

Correct. On mac it's tricky to measure memory, particularly with Activity Monitor. Enabling 'Real Memory', 'Real Private Memory', 'Real Shared Memory' by right clicking the header row will give you a better view. To be honest I have yet to figure out what the 'Memory' field is actually measuring. VSS being super high on mac is a known issue, but it has more to do with OSX not reclaiming memory unless it needs it.

Comment 8

[u285848]

Andrew, at that point it was using virtual memory.

Firefox was slowing down as the memory allocation was going up. It went into a tailspin where I could see the memory quickly growing out of control, possibly because it had dipped into virtual memory. At the time, it was just trying to load two pages at once that otherwise loaded quickly. The tabs were just "spinning" and the page was partially loaded on one and not sure how much was loaded on the other- page was blank and I didn't look into it. (Note: a FF crash report window had also been open in the background/dock, possibly from some time before, which I sent and closed after I closed FF; I don't remember that happening, so I'm not sure when or how it got there.)

After restarting Firefox, it loaded the same two pages at once without a problem and it's responsive again.

In the more recent beta versions of Firefox, it seems like it takes a long time to get back up to that state.

Eric, I have FF in safe mode now with activity monitor running with Memory, Compressed Mem, Real Mem, Private Mem, Shared Mem, and Purgeable Mem all there and ready to be screenshot when it happens again. I could also take a sample in activity monitor, or whatever else would be helpful.

It would be great if the nightly test of Firefox on macOS were to include a memory leak test that checked page loads, page cleanups, and GC happening while virtual memory in use. While it doesn't seem like it should matter and I don't think it was significantly being used until it went into a tailspin, virtual memory use in this case seemed to make the problem much worse.

In the meantime, if you need my environment for testing, I could try to setup and run some tests locally.

Comment 9

[Eric Rahm [:erahm]]

(In reply to Gary S. Weaver from comment #8)
> Andrew, at that point it was using virtual memory.
> 
> Firefox was slowing down as the memory allocation was going up. It went into
> a tailspin where I could see the memory quickly growing out of control,
> possibly because it had dipped into virtual memory. At the time, it was just
> trying to load two pages at once that otherwise loaded quickly. The tabs
> were just "spinning" and the page was partially loaded on one and not sure
> how much was loaded on the other- page was blank and I didn't look into it.
> (Note: a FF crash report window had also been open in the background/dock,
> possibly from some time before, which I sent and closed after I closed FF; I
> don't remember that happening, so I'm not sure when or how it got there.)

glandium, per comment 8 it sounds like there might be a virtual memory leak in 57 on OSX. Any pointers on tracking that down?

> After restarting Firefox, it loaded the same two pages at once without a
> problem and it's responsive again.
> 
> In the more recent beta versions of Firefox, it seems like it takes a long
> time to get back up to that state.
> 
> Eric, I have FF in safe mode now with activity monitor running with Memory,
> Compressed Mem, Real Mem, Private Mem, Shared Mem, and Purgeable Mem all
> there and ready to be screenshot when it happens again. I could also take a
> sample in activity monitor, or whatever else would be helpful.

Gary, does clicking 'Mimimize memory usage' in about:memory have any effect?

> It would be great if the nightly test of Firefox on macOS were to include a
> memory leak test that checked page loads, page cleanups, and GC happening
> while virtual memory in use. While it doesn't seem like it should matter and
> I don't think it was significantly being used until it went into a tailspin,
> virtual memory use in this case seemed to make the problem much worse.

We do have long running memory tests (AWSY), but we don't track VSS as it's super noisy. We could look into tracking it for egregious regressions.

Comment 10

[Mike Hommey [:glandium]]

(In reply to Eric Rahm [:erahm] (please no mozreview requests) from comment #9)
> (In reply to Gary S. Weaver from comment #8)
> > Andrew, at that point it was using virtual memory.
> > 
> > Firefox was slowing down as the memory allocation was going up. It went into
> > a tailspin where I could see the memory quickly growing out of control,
> > possibly because it had dipped into virtual memory. At the time, it was just
> > trying to load two pages at once that otherwise loaded quickly. The tabs
> > were just "spinning" and the page was partially loaded on one and not sure
> > how much was loaded on the other- page was blank and I didn't look into it.
> > (Note: a FF crash report window had also been open in the background/dock,
> > possibly from some time before, which I sent and closed after I closed FF; I
> > don't remember that happening, so I'm not sure when or how it got there.)
> 
> glandium, per comment 8 it sounds like there might be a virtual memory leak
> in 57 on OSX. Any pointers on tracking that down?

The output of `vmmap` when this happens would be most useful.

Comment 11

[u285848]

Attachment: vmmaplog.tgz

Comment 12

[u285848]

Attachment: Main Process

Comment 13

[u285848]

Attachment: memory-report.json.2017-10-25.gz

Comment 14

[u285848]

Attachment: gc-edges.69212.1508989754.log.tgz

Comment 15

[u285848]

Attachment: gc-edges.72650.1508989754.log.tgz

Comment 16

[u285848]

Attachment: gc-edges.67442.1508989754.log.tgz-split-aa

Comment 17

[u285848]

Attachment: gc-edges.67442.1508989754.log.tgz-split-ab

Comment 18

[u285848]

Attachment: gc-edges.67442.1508989754.log.tgz-split-ac

Comment 19

[u285848]

Attachment: gc-edges.67442.1508989754.log.tgz-split-ad

Comment 20

[u285848]

Attachment: gc-edges.67442.1508989754.log.tgz-split-ae

Comment 21

[u285848]

Attachment: gc-edges.66696.1508989754.log.tgz-split-aa

Comment 22

[u285848]

Attachment: gc-edges.66696.1508989754.log.tgz-split-ab

Comment 23

[u285848]

Attachment: cc-edges.69212.1508989754.log.tgz

Comment 24

[u285848]

Attachment: cc-edges.72650.1508989754.log.tgz

Comment 25

[u285848]

Attachment: cc-edges.67442.1508989754.log.tgz-split-aa

Comment 26

[u285848]

Attachment: cc-edges.67442.1508989754.log.tgz-split-ab

Comment 27

[u285848]

Attachment: cc-edges.66696.1508989754.log.tgz

Comment 28

[u285848]

Attached verbose vmmap output for each FF pid as well all all of the about:memory dumps.

This time just got really slow and some tabs were stuck on load, but was able to close tabs. (Was not in tailspin with memory growing.) Memory usage is high, but it reduced memory usage over an hour from close to 3GB to ~1.4GB. I'll keep an eye on it and if it gets really bad again, will try to provide more data.

Comment 29

[u285848]

Attachment: Screen Shot 2017-10-25 at 11.26.26 PM.png

Comment 30

[u285848]

Attachment: Screen Shot 2017-10-25 at 11.47.46 PM.png

Comment 31

[u285848]

Attachment: Screen Shot 2017-10-26 at 12.24.53 AM.png

Comment 32

[Mike Hommey [:glandium]]

(In reply to Gary S. Weaver from comment #28)
> Attached verbose vmmap output for each FF pid as well all all of the
> about:memory dumps.

Unfortunately, the vmmap is not for the same Firefox as the about:memory dumps one (pids are different), so we can't really compare.

Comment 33

[u285848]

@glandium

x vmmaplog/ff42694.txt - this is the only vmmap that doesn't map
x vmmaplog/ff66696.txt - matches below. use the linux "join" command to join the split files into a tar.gz file
x vmmaplog/ff67442.txt - matches below. use the linux "join" command to join the split files into a tar.gz file
x vmmaplog/ff69212.txt - matches below. use the linux "join" command to join the split files into a tar.gz file

gc_and_cc/cc-edges.66696.1508989754.log.tgz
gc_and_cc/cc-edges.67442.1508989754.log.tgz-split-aa
gc_and_cc/cc-edges.67442.1508989754.log.tgz-split-ab
gc_and_cc/cc-edges.69212.1508989754.log.tgz
gc_and_cc/cc-edges.72650.1508989754.log.tgz
gc_and_cc/gc-edges.66696.1508989754.log.tgz-split-aa
gc_and_cc/gc-edges.66696.1508989754.log.tgz-split-ab
gc_and_cc/gc-edges.67442.1508989754.log.tgz-split-aa
gc_and_cc/gc-edges.67442.1508989754.log.tgz-split-ab
gc_and_cc/gc-edges.67442.1508989754.log.tgz-split-ac
gc_and_cc/gc-edges.67442.1508989754.log.tgz-split-ad
gc_and_cc/gc-edges.67442.1508989754.log.tgz-split-ae
gc_and_cc/gc-edges.69212.1508989754.log.tgz
gc_and_cc/gc-edges.72650.1508989754.log.tgz - this is the only gc file that doesn't map to a ff pid

I think you can get what you need from that. I dumped everything that was in process memory.

Comment 34

[u285848]

btw- if there's something that you specifically need, like a series of steps that I could follow to provide you with the correct files, perhaps you could write it as a command/script/application. having users guess at this and spend an hour of their time splitting files and uploading followed by that kind of response is pretty ****.

Comment 35

[u285848]

Apologize. Closing this out as there's nothing more to do.