551999 - resource leak if I visit a page with a script similar to the one in the page that demonstrates the issue

Status: RESOLVED INCOMPLETE

(Core :: JavaScript Engine, defect)

Description

[Richard Mant]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6

A resource leak (Allocation but no deallocation) occurs over a period of several hours (though this is greatly accelerated in the attached test page) as the JavaScript in that page runs, it was originally a clock. At some critical point in the time frame, enough memory becomes allocated that it causes FireFox to crash with a call to abort().

Reproducible: Always

Steps to Reproduce:
1. Go to the page attached, checking memory usage before going there with a usage reporting utility that gives amount allocated in Kb - such as Windows' Task Manager
2. Wait 5-10 minutes, and check usage again
Actual Results:  
FireFox goes mad allocating memory even though there should be no need

Expected Results:  
FireFox has a constant memory usage and will remain usable for weeks on end with scripts like the above running

Tested initially in my default profile, then tested using the following command line (profile used is my development one; so less settings changed/messed with):

"C:\Program Files\Mozilla Firefox\firefox.exe" -no-remote -P Dev -safe-mode DX-MON.15.html

When FireFox does crash, a bug report cannot be generated due to a call to abort() being made.

Comment 1

[Richard Mant]

Attachment: Same as the URL specified, test case of the bug.

Comment 2

[Boris Zbarsky [:bzbarsky]]

Attachment: testcase that pushes a bit harder

Comment 3

[Daniel Holbert [:dholbert]]

bz's "pushes a bit harder" testcase seems to follow this pattern for me, using a debug mozilla-central build on Linux:
 - Starts out around 27 MB
 - Grows to 35 MB
 - Jumps back down to 27 MB
 - Grows to 40 MB
 - Jumps down to 27 MB
 - Grows to 77 MB
 - Jumps back down to 27 MB
 - Grows to 79 MB
 - Jumps down to 27 MB

This all happened over the course of maybe two minutes.

Over the following two minutes, memory usage never grew beyond ~80 MB, though.  It kept growing up to the ~79 MB limit and then jumping back down to 27 MB (or 56ish Mb in a few cases)  It seems to "plateau" (with periodic garbage collection to jump back down to "ground level") So, I don't think I'm seeing any leaking behavior on my build, FWIW.

Comment 4

[Boris Zbarsky [:bzbarsky]]

So I just tried running the testcase I attached in a debug Linux m-c build for a bit, with a breakpoint in js_GC and logging the number of nsGenericElements around.  I do see GC happening.  I don't see the number of nsGenericElements increasing.  When GC is hit, here are the before/after RSS numbers:

160mb -> 79mb 
164mb -> 86mb
172mb -> 90mb
103mb -> 88mb
176mb -> 94mb
177mb -> 89mb
177mb -> 88mb

That's running for about an hour or so over here.

Comment 5

[Mike Shaver (:shaver emeritus)]

Has anyone tested it on 3.6 on Windows?  The date code has platform-specific calls in a couple of places, which might not be handling memory correctly, or it might be something we fixed since 3.6.  I can test in a little while if nobody else does.

Comment 6

[Richard Mant]

To those testing on Linux: Please note that I submitted this as a bug that I know works on Windows. I think (like the poster above) that this is a platform-specific bug. I can easily reproduce it under XP on either 3.5.8 or 3.6; I don't think that 3.5.7 has this bug.

Comment 7

[Richard Mant]

Ok, contrary to my thinking, I can reproduce this bug on 3.5.7 too.. I will go to 3.5.0 and test, but not back further - If memory serves me right, 3.0 and 3.5 are technically different architectures, especially around the JavaScript area..

Also, I have noticed that as with bz and dholbert, on bz's 'pushing harder' version, GC does take place (it does for my version too, but at an incredibly slower rate) but it is too little too late. I think that (although I don't think the GC is naive) the Garbage Collector under Windows is only getting triggered after a certain amount of user JS (as opposed to Chrome JS) has been run.. like, say, 1 million lines.. but this is only an observation - I don't know if this is actually true.

Consider this: I tend to run about 100 tabs (long story as to why), if this bug is being reproduced by all of them and GC is not taking place fast enough, what's going to happen? FireFox will crash due to going into an out of memory state. So if we can work out why so much memory is being allocated and then not being freed very fast.. well, you get my drift.

I'll also test if running 5 tabs causes the amount of memory leaked to follow the following formula where X is the amount leaked for one and Y is the total leaked: 5 * X = Y

Comment 8

[Richard Mant]

Just tested that formula, and it holds true.. 5 tabs of the test-case I put together provided a leak 5 times as big, so, if we say that FireFox is leaking about 20Mb in 30 seconds to 1 minute with just 1 instance, I just had a 100Mb leak occur with 5 instances and 30 seconds.
Only after that 30 seconds did garbage collection kick in and that released 75% of the memory associated with the leak, which meant next cycle gave rise to a total of 125Mb leaked, and this keeps happening, so the overall leak keeps getting bigger and bigger, despite GC.

Comment 9

[Richard Mant]

Been a while since anything was said on this bug. I have just started a time of exams in my college/uni life, so I cannot perform any more tests for the moment. If another of you would like to try regression testing this bug, I'd like to know the results as I hope it isn't just my configuration that is causing a problem, or something like that.

Comment 10

[Daniel Holbert [:dholbert]]

(In reply to comment #8)
> I just had a 100Mb
> leak occur with 5 instances and 30 seconds.
> Only after that 30 seconds did garbage collection kick in and that released 75%
> of the memory associated with the leak,

Just to clarify your language here: that's 100Mb **allocated**, not leaked.  75% of that allocated memory when the GC kicks in, which means only 25Mb is (potentially) leaked.

(Of course, if this pattern continues as you say it does, then over 100Mb would be eventually leaked -- but "100Mb leaked in 30 seconds" is inaccurate.)

Comment 11

[Daniel Holbert [:dholbert]]

(In reply to comment #10)
> 75% of that allocated memory when the GC kicks in, which means only 25Mb is
> (potentially) leaked.

sorry - meant to say "75% of that allocated memory _gets freed_ when the [etc]"

Comment 12

[Richard Mant]

Ok, perhaps it was a little inaccurate, but it would not take long for that to be permanently greater than 100Mb allocated to FireFox.

On a slightly different note to that, now that the Mozilla website seems to be back up, I have begun trying to get a bloat-log out for Boris' Pushing Harder version of the test page. (Needed to look up how to get bloat-logging enabled, hence the need for the Mozilla site) Just about to fire up 3.6 built in debug mode on XP to see what I can (or can't..) find.

Comment 13

[Richard Mant]

Attachment: Bloat-log from a debug test-run on XP

Just run a fresh build (debug) of Namoroka 3.5 and after using Boris' Pushing Harder test with Bloat-Logging, have a good log of what happened. See the attached stats.

Pushing Harder test was run for 5 minutes, with Namoroka left open with just about:bloat for a further 30.

Comment 14

[Richard Mant]

erm, I said 3.5 there, I meant 3.6 (sorry for the SPAM..)