902913 - The Garbage Collector misses stuff that should be garbage collected (and are not DOM related)

Status: RESOLVED DUPLICATE of bug 753203

(Core :: JavaScript Engine, defect)

Description

[Xavier]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0 (Beta/Release)
Build ID: 20130730113002

Steps to reproduce:

Using a chained list of objects, I came accross a situation where objects should be collected, and aren't.

I've stripped down most of the code that was not useful in reproducing the issue, which now makes it look useless, but still shows the issue.

var line = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
+ "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
+ "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
+ "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
+ "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";

var million = 1000000;
var maxCount = 100 * million;
var obj, previous;
for(var idx=0; idx < maxCount; idx ++) {
  obj = {
    line1: line, line2: line, line3: line,
    line4: line, line5: line, line6: line, line7: line,
    line8: line, line9: line, next: null
  };
  if(previous) {
    previous.next = obj;
  }
  previous = obj;
}


Actual results:

If you run this script in spidermonky shell (windows 64b), it will quickly throw an 'out of memory' exception.
I expected that the first object created should be available to GC at the end of the second loop execution, when 'previous' is reaffected with the newly created 'obj', leaving this new 'obj' only referenced by 'previous' (and no longer by the previous value of 'previous.next')

Of course this is useless here, since I removed the code that would make it useful.

I've also ran it with explicit gc() calls every X iterations, which slows things down considerably but does not solve the issue.
I've also crashed Firefox with it.

To make sure I wasn't overlooking an 'obvious' cycling that might lure the GC, I've tested it against nodejs, rhino, tamarin and cscript, while monitoring their memory usage, and all were handling it fine. I've also ran a 'full-featured' version in which the goal is to keep a maximum of N chained objects at anytime by dereferencing the older one, and checking after the loop completes that I had N objects chained.
Just to say that not only did it not crash, it also worked as expected, which might not be obvious with the 'light' version.




Expected results:

It should have ended normally.

Comment 1

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

This appears to be an unfortunate result of conservative stack scanning, which is described in the first portion of this article:
  https://blog.mozilla.org/javascript/2013/07/18/clawing-our-way-back-to-precision/

I'm guessing that when we transition from the interpreter to the baseline JIT, we leave a pointer to whatever element was being constructed at that time on the C stack. In each later GC, we find that pointer and then mark every object that has been constructed since then by following the chain of "next" pointers.

It's possible that we could try to fix this by finding the pointer from the interpreter stack and nulling it out explicitly, but I'm skeptical that would really be much help. We've already done that once before and these are still popping up.

The work to eliminate conservative stack scanning is underway, as the blog entry describes, and that will completely solve this problem. Until then, I can try to suggest workarounds if you tell me more about how your actual code works.

I'm sorry this happened. We've been working to solve problems like this for over a year now, but it's difficult.

Comment 2

[Xavier]

Thanks for the explanation, and I'm sure it's no easy issue.

As a workaround, I've used a cyclic array (when it's full I start over from its begining and keep a pointer to the last entry) to store my N objects.

The original code showing the issue was little more than the one posted with one more var to keep track of the very first object created, that I would later (once N was reached)set to the object contained in its 'next' field, hoping to de-reference and free the first one, and so on. And of course some filtering because not all object would qualify to be kept in the N subset (all having different member values, of course)

Since performance is important speedwise, if you have a workaround that's faster than my cyclic array, I'd like to try it :)
Thanks again.

Comment 3

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

> Since performance is important speedwise, if you have a workaround that's faster than my
> cyclic array, I'd like to try it :)

That should be quite efficient.

Comment 4

[Andrew McCreight [:mccr8]]

The GC uses precise stack scanning now, so maybe this isn't an issue any more?

Comment 5

[Terrence Cole [:terrence]]

Only on desktop, but we're rapidly approaching having it universally enabled. I think we're okay to close this.

Comment 6

[Xavier]

Still having this issue with Firefox 52.0.2 (32 bits)
I stopped when 3Gb of memory where being used, didn't wait until the crash, though