745448 - Weird gc behavior on Linux 64bits PGO

Status: RESOLVED WONTFIX

(Core :: JavaScript Engine, defect)

Description

[David Teller [:Yoric] - still alive but not very active]

Attachment: Test case

I am currently investigating bug 745233 and I have a test case in which the gc shows a different behavior when we are running PGO.

I am allocating 100 data structures with finalizers (using CDataFinalizer, see bug 720771), without any reference to these data structures. I then run the gc.

In non-PGO mode, all 100 data structures are gc-ed and finalized immediately. In Linux 64 bits PGO mode, only 99 data structures are gc-ed and finalized immediately. The 100th data structure is gc-ed only at the end of the test.

This behavior is surprising and may indicate the possibility of a resource leak. In addition, it prevents my bug 720771 from landing :)

Comment 1

[David Teller [:Yoric] - still alive but not very active]

Currently running this through the TryServer to obtain confirmation.

Comment 2

[Brian Hackett [Laid off!]]

With the conservative stack scanner the objects that are traced on each collection are sensitive to the compiler, optimization level, and all sorts of other things.  We're slowly moving back to an exact scanner which will behave more predictably (necessary for moving generational/compacting GC), e.g. bug 714647 which landed yesterday is a big step in that direction.

Comment 3

[David Teller [:Yoric] - still alive but not very active]

(In reply to Brian Hackett (:bhackett) from comment #2)
> With the conservative stack scanner the objects that are traced on each
> collection are sensitive to the compiler, optimization level, and all sorts
> of other things.  We're slowly moving back to an exact scanner which will
> behave more predictably (necessary for moving generational/compacting GC),
> e.g. bug 714647 which landed yesterday is a big step in that direction.

But isn't it strange that one of the structures is never collected before the end of the test?

Comment 4

[Brian Hackett [Laid off!]]

No, it isn't strange at all.  There could be a C stack location pointing to the object which is not overwritten at any point in the test, and keeps that reference until the final GC.  Tests should not currently assume anything about when objects will be collected or in which order.

Comment 5

[David Teller [:Yoric] - still alive but not very active]

(In reply to Brian Hackett (:bhackett) from comment #4)
> No, it isn't strange at all.  There could be a C stack location pointing to
> the object which is not overwritten at any point in the test, and keeps that
> reference until the final GC.  Tests should not currently assume anything
> about when objects will be collected or in which order.

Makes it a little hard to write finalization tests :/

Ah well, I guess I will have to rewrite my test suite to ensure that every run takes place in a distinct script.

Comment 6

[David Teller [:Yoric] - still alive but not very active]

Note: difference in behavior can be seen with

- https://tbpl.mozilla.org/?tree=Try&rev=07e95f688aac (no PGO, all 100 items are collected)

- https://tbpl.mozilla.org/?tree=Try&rev=810ad1268bd2 (PGO, only 99 items are collected)

Comment 7

[David Teller [:Yoric] - still alive but not very active]

Brian, I have experimented further and it looks like the first value (and only the first value) I create with a finalizer is only gc-ed at the end of the test. Note that I invoke Components.utils.gc() hundreds of times, run dozens of loops and that I have left the scope in which this reference-less value was created almost immediately after creating the value. Is this expected?

Regardless, I am currently rewriting my test suite using chrome workers to isolate the memory of each test.

Comment 8

[Terrence Cole [:terrence]]

This is exactly the sort of weirdness that you should expect with conservative stack scanning.

Comment 9

[David Teller [:Yoric] - still alive but not very active]

(In reply to Terrence Cole [:terrence] from comment #8)
> This is exactly the sort of weirdness that you should expect with
> conservative stack scanning.

Oh. I guess this means "conservative" as in Boehm-style "anything-could be a pointer" gc?

Regardless, I have rewritten my tests and now obtain the right results, so I will probably mark this bug as WONTFIX.

Comment 10

[Terrence Cole [:terrence]]

(In reply to David Rajchenbach Teller [:Yoric] from comment #9)
> Oh. I guess this means "conservative" as in Boehm-style "anything-could be a
> pointer" gc?

Precisely. :-)

> Regardless, I have rewritten my tests and now obtain the right results, so I
> will probably mark this bug as WONTFIX.

Excellent.  It is an extremely bad idea for client code to depend on GC behavior: even without conservative scanning, unrelated engine changes can change the behavior of the GC.  For example, what if someone adds an object cache somewhere deep in the JS engine?  If some of your test objects happened to land in it, your test would break even without conservative scanning.