532209 - Garbage collection not running automatically

Status: RESOLVED DUPLICATE of bug 522856

(Core :: JavaScript Engine, defect)

Description

[Cal Heldenbrand]

User-Agent:       Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.15) Gecko/2009102815 Ubuntu/9.04 (jaunty) Firefox/3.0.15
Build Identifier: 1.8.5 from Mercurial as of 20091201

I've been playing with the latest 1.8.1 source, and it appears that something is wrong with the garbage collection threshold specified in JS_NewRuntime().  Here's some quick example code:

for ( var i = 0 ; i < 1000 ; i++ )
{
        print(i);
        //gc();
        obj = new Array();
        for ( var j = 0 ; j < 100000 ; j++ )
                obj[j] = "1234567890";
}

I'm using the latest source updated today (12/01/09) compiled in Linux, and executed with the js shell.  If I let this run to completion with 1.8.1 it consumes around 1GB of memory, and the js shell should kick in at 64MB.  This same code run with release 1.7 runs as expected -- it jumps up to 64MB, then cleans up after itself.

If I explicitly call gc() in the code above, then it seems to work fine in 1.8.1.  (Consumes around 2MB of memory)

Thank you!

Reproducible: Always

Actual Results:  
The script consumes around 1GB of memory

Expected Results:  
Compared to JS 1.7, it should stay at around 64MB.

Comment 1

[David Mandelin [:dmandelin]]

Attachment: Patch

Bug 522856 will solve this. But I made a 1-liner that should at least correct this problem.

Comment 2

[Igor Bukanov]

Comment on attachment 415526 [details] [diff] [review]
Patch

This is fine as a small fix. But we should remember to remove it after addressing the bug 522856.

Comment 3

[Cal Heldenbrand]

I apologize for the duplicate issue, I didn't search for the right terms.  I applied that one liner patch to jsarray.cpp and I still seem to get 1GB of memory consumption on my test.

Thanks for looking at this issue!

--Cal

Comment 4

[David Mandelin [:dmandelin]]

(In reply to comment #3)
> I apologize for the duplicate issue, I didn't search for the right terms.  I
> applied that one liner patch to jsarray.cpp and I still seem to get 1GB of
> memory consumption on my test.

I'm really surprised this patch doesn't fix the issue for you. But given that it doesn't, and the patch was supposed to be a quick fix, I'm inclined not to land it. If you can wait for bug 522856, then we can probably just dup to that.

Comment 5

[Cal Heldenbrand]

David,

I double checked this again with the js shell with SpiderMonkey and it does work as expected.  My application is based off of js.c from 1.7, but I ported over most of the changes for 1.8. 

I can do a big diff again and try to find what is different in my app, but do you know off hand what other functions might affect this?  (My application consumes 1GB of memory, but the js shell works fine now)

Thanks for any help!

--Cal

Comment 6

[David Mandelin [:dmandelin]]

(In reply to comment #5)
> David,
> 
> I double checked this again with the js shell with SpiderMonkey and it does
> work as expected.  My application is based off of js.c from 1.7, but I ported
> over most of the changes for 1.8. 

I'm a little confused now because of all the different dimensions of variations in play here: JS version (1.7, 1.8, 1.8+my patch), JS build (shell or your embedded app), and possibly test case (your test case above or some other program). Can you tell me exactly which combinations do and don't cause mem blowup for you?

> I can do a big diff again and try to find what is different in my app, but do
> you know off hand what other functions might affect this?  (My application
> consumes 1GB of memory, but the js shell works fine now)

The bug that my patch fixes is one of a class of bugs where memory allocated by JS_realloc is not added to the counter that is used to decide when to GC. In the JS shell, your test case uses dense arrays and calls JS_realloc at a certain point. It is possible that in some other configuration a different JS_realloc is being used, or that you are triggering an entirely different bug.

Comment 7

[Cal Heldenbrand]

(In reply to comment #6)
> I'm a little confused now because of all the different dimensions of variations
> in play here: JS version (1.7, 1.8, 1.8+my patch), JS build (shell or your
> embedded app), and possibly test case (your test case above or some other
> program). Can you tell me exactly which combinations do and don't cause mem
> blowup for you?

Sorry, I should have given a little more detail.  Since the same problem happened on the js shell as my embedded app, I assumed they would be due to a common bug.

My application is a CGI app with SM embedded to add server side javascript.  (linked against libmozjs.so)  I started all of it by using js.c in SM 1.7 as an example and adding a bunch of extra objects.  (database connectivity, etc)

I'm comparing a few differences in /src/shell/js.cpp in 1.8 with my hacked up version of 1.7, and porting over what looks important.  (And reading through the API as well)

My application fires up and shuts down within an average of 500ms... but there could be a possibility of a few hits coming in on the same server that could crush a machine.  So it's a toss up -- speed without garbage collection, or the occasional dead server.

> certain point. It is possible that in some other configuration a different
> JS_realloc is being used, or that you are triggering an entirely different bug.

I actually was using the same javascript code in my embedded app as I provided.  (That's why I thought they would be the same bug)

I hope this gives enough background.  Let me know if you need any more information, and if you might know what API settings could cause this.

I'll just resume studying the new js shell to see if I'm missing something in my app.

Thank you,

--Cal

Comment 8

[David Mandelin [:dmandelin]]

(In reply to comment #7)
> I'll just resume studying the new js shell to see if I'm missing something in
> my app.

OK, it looks like immediate action on my part is not required (let me know if I'm mistaken). The APIs for tuning GC scheduling have changed, and studying the JS shell is probably the best way to see how the new stuff works.

Comment 9

[Cal Heldenbrand]

Yes, the patch proved that the leak is fixed.  Thanks for your help David!  I'll mark this as a dup of bug 522856.

Comment 10

[Cal Heldenbrand]

David,

I think I found another variable -- I forgot that js shell has JIT turned off by default.  The javascript example above works correctly when JIT is turned off, but when run with "js -j example.js" it will use 1GB of memory, the same as my application.

Could this possibly be a bug relating to memory management in JIT?

Thank you,

--Cal

Comment 11

[David Mandelin [:dmandelin]]

(In reply to comment #10)
> David,
> 
> I think I found another variable -- I forgot that js shell has JIT turned off
> by default.  The javascript example above works correctly when JIT is turned
> off, but when run with "js -j example.js" it will use 1GB of memory, the same
> as my application.
> 
> Could this possibly be a bug relating to memory management in JIT?

I just asked, and for now we don't run the GC if we are on trace, so the memory doesn't get reclaimed until we leave trace. If both loops trace, then that could be a while.