803019 - "Assertion failure: grayRoots.empty()" with schedulegc and gczeal

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Jesse Ruderman]

Attachment: testcase

1. Install https://www.squarefree.com/extensions/domFuzzLite3.xpi
2. Load the testcase

Assertion failure: grayRoots.empty(), at js/src/jsgc.cpp:2030

Comment 1

[Jesse Ruderman]

Attachment: stack

Comment 2

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

Attachment: patch

The problem here is that we are only asked to collect the atoms compartment. However, we can only collect the atoms compartment if we're collecting everything else. So effectively we're not collecting any compartments.

I considered fixing the testing code so that we aren't allowed to schedule a GC on only the atoms compartment. However, it's sort of awkward to do that. Also, I think it might make sense to handle this case correctly anyway. So the patch just aborts the GC if no compartments need to be collected.

While checking that this is an okay thing to do, I noticed that the JSGC_END callback does ClearGCBeforeCC unconditionally. However, that flag is basically saying that some of the gray mark bits are in an undefined state. We should only clear it if the GC has successfully reset the gray bits for all compartments. And that only happens in a full GC. So I fixed that.

Comment 3

[Jon Coppeard (:jonco)]

Comment on attachment 686878 [details] [diff] [review]
patch

Review of attachment 686878 [details] [diff] [review]:
-----------------------------------------------------------------

Yes that looks good.

Comment 4

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/d8c788ff2545

Comment 5

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/d8c788ff2545