38942 - js_GC can lead to memory exhaustion when low on memory

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect, P3)

Description

[Chris Nicotra]

From Bugzilla Helper:
User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows 98)
BuildID:    

Since the js_GC routine calls the same JS_ARENA_ALLOC as the rest of the code, 
it is most likely going to fail to allocate the memory it needs to perform the 
garbage collection if it is called after a failed memory allocation. The error 
handling when this happens is not freeing stuff, but it is forgetting about it. 
In my application this caused a major memory leak. My solution was to get a 
global state variable indicating I was in the garbage collector and have the 
lower level memory routines use a different pool while in the garbage collector.

Reproducible: Always
Steps to Reproduce:
Call the js_GC routine when the malloc routine can not allocate any more memory 
and there are items to be garbage collected.

Actual Results:  The items that would have been garbage collected are forgotten 
and the memory is leaked.

Expected Results:  The garbage collector would at least leave the items in a 
state where they could be garbage collected later. Ideally, memory would either 
be in reserve or come from a different pool so that the garbage collector would 
never fail due to low memory.

Comment 1

[Gervase Markham [:gerv]]

Needs to be assessed by developer; confirming.

Gerv

Comment 2

[Phil Schwartau]

Reassigning to Patrick as per meeting; cc'ing Brendan

Comment 3

[Brendan Eich [:brendan]]

I don't see how GC-things to be finalized, if there is an error allocating 
tempPool space from the malloc heap to hold the finalization records, are 
forgotten and permanently leaked.  Chris, can you say more?  The relevant 
allocation is at http://lxr.mozilla.org/mozilla/source/js/src/jsgc.c#1067, and 
if it fails, the GC goes to out, leaving the unswept/unfinalized garbage to be 
found by the next GC (if it doesn't run out of memory too).

Maybe that's the problem: you are counting on the JS GC to recover malloc space, 
and it's not finalizing what it could (the GC-things remembered by finalization 
records already allocated from tempPool, if any).  Attached is my patch to 
correct this pessimism.  Using a dedicated piece of memory to hold finalization 
records would do even better.

Patrick, I have other changes (js1.5 bugs on my list) coming here.  I'm happy to 
take it from you, if you like.

/be

Comment 4

[Brendan Eich [:brendan]]

Attachment: small fix to finalize what we can each time the GC runs

Comment 5

[Patrick C. Beard]

Reassigning to brendan. Should this go into beta3?

Comment 6

[Brendan Eich [:brendan]]

Yes, this should go into js1.5 final too; at least one Mozilla community member 
needs it now.

/be

Comment 7

[Brendan Eich [:brendan]]

The patch in bug 49816 fixes this bug too.

/be

Comment 8

[Brendan Eich [:brendan]]

Fix is in -- I dropped the big one!

/be