1141242 - [escalate] Continuously refreshing a tab after an interval leads to high memory consumption

Status: RESOLVED INCOMPLETE

(Core :: JavaScript: GC, defect)

Description

[mpawar.splunk]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:36.0) Gecko/20100101 Firefox/36.0
Build ID: 20150222232811

Steps to reproduce:

Environment: MAC OSX 10.10.2 Firefox 36

Firefox keeps consuming lot of memory when you keep refreshing a tab at some interval.

- Start the firefox browser
- Visited gmail.com and logged into my account
- Monitored firefox memory consumption rate in the "activity monitor" application of MAC.
- Refreshed the page couple of times and notice the increase in the memory consumption rate.
- I work at Splunk and i found the similar behavior when the Splunk dashboards (with or without charts) were refreshed at specific interval. The memory consumption rate was really high and this happens only in Firefox browser.


Actual results:

For the gmail testing:
Initially, memory consumption was about 400MB. I refreshed the page after 10 seconds and it went to 580MB. Again i refreshed after 10 seconds and this time it was 690MB. Finally, when i refreshed 3rd time after 10 seconds, it was showing as 800MB.

For the splunk dashboards testing: i had three 3 dashboards open in 3 different firefox tabs and these tabs were refreshed at 30 secs interval. Eventually, the firefox crashed after the high memory consumption (around 2GB).

It's not a memory leak because at a point even if the memory consumption is really high, it would stay at that rate (neither increase nor decrease)


Expected results:

Even if firefox consumes lots of memory, the memory should go down after some time which never happens. Even if you close the tab and open a new blank tab, you would see that the memory consumption rates is still not reduced quickly.

Comment 1

[mpawar.splunk]

- I tried playing with memory profiling and about:memory configurations but that didn't help.
- I also tried building firefox nightly with dtrace configuration to see if i get something but unfortunately that also didn't help.

Comment 2

[mpawar.splunk]

@Firefox support: I work in San Francisco and i know that you have office here too.
If you want, i can also come to your office and show the issue. It would be great if you can help me with this problem because right now we have to tell our clients to use any other browser except Firefox. Thanks.

Comment 3

[[:Aleksej]]

Thanks for the report.  Before somebody does something with it (https://wiki.mozilla.org/Performance/MemShrink), maybe https://developer.mozilla.org/en-US/docs/Zombie_compartments is helpful.

Comment 4

[Timothy Nikkel (:tnikkel)]

(In reply to mpawar.splunk from comment #1)
> - I tried playing with memory profiling and about:memory configurations but
> that didn't help.

Could you upload an about:memory memory report here? That way we can at least see what is using the memory. It's a starting point.

Comment 5

[mpawar.splunk]

Attachment: memory report

Timothy Nikkel,

Here is the memory report of firefox where the memory consumption rate was 1.07GB with three tabs open on which same splunk dashboard was getting refreshed after every 30 seconds.

Comment 6

[Timothy Nikkel (:tnikkel)]

Most memory seems to be used by js related things. Nick, can you provide any insight?

Comment 7

[Nicholas Nethercote [inactive]]

What do you mean by "refresh"? Do you mean hit the reload button (circle-shaped arrow) in the address bar?

When I reload gmail repeatedly (with that button) I get an up-and-down pattern -- memory increases on each reload, and then eventually drops again once the GC kicks in. I would expect similar behaviour from other sites.

Comment 8

[mpawar.splunk]

Refresh behaves similar to browser tab reload. The javascript code which executes for this is location.reload();

Comment 9

[Timothy Nikkel (:tnikkel)]

Like Nick says, refreshing a page is expected to increase the memory usage some, but it should level off shortly.

Can you load up whatever tabs, save a memory report, and then let them refresh until the memory use gets ridiculous and then save another memory report? Then we can diff them to see whats going on.

Comment 10

[mpawar.splunk]

Attachment: Memory report for initial state

Memory report of when the 3 tabs with splunk dashboard were loaded for the first time. The memory consumed by firefox at this time was around 650MB.

Comment 11

[mpawar.splunk]

Attachment: Memory report for later state

Memory report of when the 3 tabs with splunk dashboard were refreshed at every 30 seconds for around 5 minutes. At this time, the memory consumed by firefox at this time was around 1.35GB.

Comment 12

[mpawar.splunk]

If you want, you can go to http://www.splunk.com/page/previous_releases and download a free version of any release starting from 6.0 version and reproduce the issue. You just have to create a simple dashboard in Splunk which will reload at every 30 seconds and open that dashboard in 3 tabs. Let me know if you need more help on how to create dashboard in Splunk and reproduce the issue.

Thanks for all the help.

Comment 13

[Timothy Nikkel (:tnikkel)]

What happens if you leave it running for longer than five minutes? Does it just keep growing forever? Does it plateau?

Comment 14

[mpawar.splunk]

No, it does not keep growing forever. For e.g. after 5 minutes if the memory consumption rate is 1.5gb then it would stay around that. Something like 1.45gb/1.5gb/1.55gb, but won't keep increasing forever. Its not a memory leak.

Comment 15

[mpawar.splunk]

Any update on this? Thanks.

Comment 16

[Timothy Nikkel (:tnikkel)]

Maybe Nick can see if we are using more memory then needed here.

Comment 17

[Timothy Nikkel (:tnikkel)]

How much memory do the other browsers use on this workload for comparison?

Comment 18

[mpawar.splunk]

This is how memory is used by other browsers.

Google chrome:
- Initial load: 446MB
- After 5 mins: 630MB

Safari:
- Initial load: 270MB
- After 5 mins: 280MB

Internet explorer 11:
- Initial load: 150MB
- After 5 mins: 350MB

Comment 19

[mpawar.splunk]

Can i get an update on this please? I want to escalate the issue and i am not sure whether adding "escalate" in the problem statement would do that. Our clients are not able to use firefox browser due to this issue.

Thanks.

Comment 20

[Kartikaya Gupta (email:[e-mail])]

Nick appears to be on PTO until the end of the month, adding :mccr8 instead.

Comment 21

[Andrew McCreight [:mccr8]]

Looking at the diff between the before and after report, there's another 200MB of explicit memory.  The resident increased by about 470MB.  I don't know why that is so much more, or if that's normal or not.

Of the explicit difference, 55MB is unused-gc-things, and 26MB is strings.  (In terms of strings, in the before there are 10's of thousands of copies of various strings like "[object Number]".)  There's another 32MB of heap-overhead, and about 20MB of network/cache2.  There's also 62MB of various JS things, like objects, arrays and functions.

So, about half of the increase is kind of heap fragmentation stuff probably caused by a lot of churn of things in memory.  I'll try to reproduce locally to see if I can figure out why the JS stuff is alive.

Comment 22

[mpawar.splunk]

There is a property called javascript.options.mem.high_water_mark in about:config. I'd set it to 10 and noticed that the amount of memory consumed by Firefox was dropped down to half as compared to using with default value 128.

I was running Splunk dashboard on 3 tabs of firefox which would reload after every 30 seconds and this is what i got:
- when javascript.options.mem.high_water_mark=128 : After 2.5 minutes, the memory consumption rate was 1.98GB
- when javascript.options.mem.high_water_mark=10: Even after 5 minutes, the memory consumption rate was 975MB

Are there any side effects of setting javascript.options.mem.high_water_mark to 10?

Comment 23

[Andrew McCreight [:mccr8]]

(In reply to mpawar.splunk from comment #22)
> There is a property called javascript.options.mem.high_water_mark in
> about:config. I'd set it to 10 and noticed that the amount of memory
> consumed by Firefox was dropped down to half as compared to using with
> default value 128.

That's interesting.  It is consistent with how the about:memory report looks like there's a lot of fragmentation.  With a lower water mark, we'll end up GCing more often, and things won't bloat up quite so much.  I wonder if other browsers have different collection triggers.

> Are there any side effects of setting javascript.options.mem.high_water_mark to 10?

You'll get more garbage collections, which might make things more stuttery when you have many tabs open.  10 seems a little low.  Maybe try something like 32 or 64?  FWIW, we seem to use a value of 6 on our FirefoxOS phones (which can have total memory as low as 128MB) and 16 or 32 on Android.
http://mxr.mozilla.org/mozilla-central/search?string=javascript.options.mem.high_water_mark&filter=[Jj]avascript.options.mem.high_water_mark

Comment 24

[mpawar.splunk]

Attachment: About memory - malloc heap

The javascript.options.mem.high_water_mark setting takes care of gc-heap but i also see that malloc-heap keeps increasing. I would like to know why does this increases if i am refreshing the same page again. I would expect that during refresh, the current page should be discarded completely and it should load a fresh version of same page with consuming same amount of memory. Is there any setting similar to javascript.options.mem.high_water_mark which i can use it for malloc-heap?

Comment 25

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

This does not seem related to the Memory developer tool.

Comment 26

[:Gijs (he/him)]

Andrew, is there anything we can do about this as currently filed and/or do we need more information?

Comment 27

[Andrew McCreight [:mccr8]]

The GC is not triggering enough to reduce memory usage, it sounds like. For the Gmail scenario in comment 0, it is isn't too surprising that a page wouldn't get cleaned up after 10 seconds. Somebody would have to look at the splunk dashboards thing to see if they are allocating something that isn't reported to the GC or whatever.

Comment 28

[Areg Vrtanesyan]

Hi

I can easily trigger this using Zabbix page autorefresh and also Jira Reindex page.

My firefox page is eating about 1Gb RAM and forcing me to restart browser or wait until it will crash itself.

Regards,
Areg

Comment 29

[Areg Vrtanesyan]

https://jira.atlassian.com/browse/JRA-62024

Comment 30

[ovidiu boca[:Ovidiu]]

I have tested on Mac OS X 10.10 with FF 47 and FF Nightly 50.0a1 (2016-07-25) and I have about 240-270 MB of RAM consumption. I used the link from comment 29. 

Areg do you have add-ons? Can you please test this with new profile or in safe mode?

Comment 31

[Areg Vrtanesyan]

I have tested this with and without Add-Ons in FF.
Atlassian Support was able to reproduce this with smaller instance that mine one.
Reindex on my side takes couple of hours to complete and periodical refresh accumulates too much RAM during refresh.

Comment 32

[Areg Vrtanesyan]

Forgot to mention that I am using FF 48.0b9 64bit Beta channel one.

Comment 33

[brian]

A fairly simple Jira webpage is taking about 500MB in FF65. Atlassian, the makers of Jira, think it's caused by this bug and referred me here. Their ticket is https://jira.atlassian.com/browse/JRACLOUD-70285
Would love to see this fixed!

Comment 34

[qvazar]

Surprised this is P3. Lot's of apps like Jira getting affected, hard to justify using FF if it just consumes all the available memory.

Comment 35

[Timothy Nikkel (:tnikkel)]

Since Jira is pointing people to this bug as a landing ground for memory problems, maybe we should take another lot here? Clearing priority to get another look.

Comment 36

[Jozef Mrkva]

Hello, I am still experiencing this bug espetially with the new FB Messenger and YouTube, but aswell with my new employer's system and it's quiet annoying having to close and reopen tabs every so often (actually quiet often lately).
I have not posted any comments on bugzilla yet, and I do not generally know much about how does it here works, I am just an occasional reader, but I have read somewhere that certain bugs can be forgotten or rather burried by other bugs which do get lots of new comments, so I decided to write here with the hope of someone seeing this and potentially addressing it.

Comment 37

[Timothy Nikkel (:tnikkel)]

It might be better to file a new bug with the problem you are seeing. Please include your about:support and a memory report of when you see high memory usage as well as your best attempt at some steps to reproduce. Thanks

Comment 38

[BugBot [:suhaib / :marco/ :calixte]]

Redirect a needinfo that is pending on an inactive user to the triage owner.
:sdetar, since the bug has high severity, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 39

[:Gijs (he/him)]

The original reporter's account is disabled, and new issues probably don't benefit from being linked to a 7-year old one, so I'm going to close this out to avoid confusion going forward. The JIRA ticket on Atlassian's side is also closed.