734763 - "Assertion failure: rt->gcMode != JSGC_MODE_GLOBAL" with certain GC prefs and forceShrinkingGC

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Jesse Ruderman]

Attachment: testcase

1. Install the new version of 
  https://www.squarefree.com/extensions/domFuzzLite3.xpi

2. Set

user_pref("javascript.options.mem.gc_incremental", false);
user_pref("javascript.options.mem.gc_per_compartment", false);

3. Load the testcase.

Result:
Assertion failure: rt->gcMode != JSGC_MODE_GLOBAL, at jsgc.cpp:3600


Based on the stack trace, I suspect this is a regression from the following chunk of bug 730447's patch:

@ MaybeGC
+    if (comp->gcMallocAndFreeBytes >= comp->gcTriggerMallocAndFreeBytes) {
+        GCSlice(cx, comp, GC_NORMAL, gcreason::MAYBEGC);
+        return;
+    }

Comment 1

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

Attachment: fix

This fixes two problems:
1. We're triggering a compartment GC when we're not supposed to.
2. We're triggering a GC when malloc bytes and trigger malloc bytes are both 0.

Comment 2

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/ee1534c3a8e7

Comment 3

[Marco Bonardo [:mak]]

https://hg.mozilla.org/mozilla-central/rev/ee1534c3a8e7