Currently we add GCThings of kind JSTRACE_OBJECT and JSTRACE_XML to the CC graph. We only care about JSTRACE_OBJECT, but JSTRACE_XML things can hold JSTRACE_OBJECT things. Recursively traversing JSTRACE_XML could overflow the stack so we dealt with that by just making it part of the CC graph. It looks like the same stack overflows could happen to JSTRACE_SHAPE things. Another issue is that UnmarkGrayChildren uses ADD_TO_CC (which currently means JSTRACE_OBJECT or JSTRACE_XML things) to decide when to stop recursively marking. I think that means that we wouldn't "unmark gray" the JSObjects held by shapes held by an object. Note that UnmarkGrayChildren probably suffers from the stack overflow problem that we tried to avoid by adding JSTRACE_XML things to the CC graph. We could avoid adding both JSTRACE_XML and JSTRACE_SHAPE things to the CC graph and fix UnmarkGrayChildren by using a different scheme to avoid stack overflow, maybe similar to the one the JS GC uses.
Overflows in UnmarkGrayChildren are no longer an issue, because of bug 660778, which watches for overflow, and requires a GC before the next CC if there is overflow . The second issue remains, wherein gray objects held by shapes won't be ungrayed properly. That could probably be handled even without adding shapes to the CC graph, by only skipping strings, instead of all non-AddToCCKind things. We could even add a new AcyclicKind or something that would only return true for JSTRACE_STRING. The drawback of this is that it would cause unmarkgray to take longer, and possibly slow down the browser because of causing more overflows, leading to more GCs .  I wonder if that means we can stop adding JSTRACE_XML to the CC graph? I wonder how common XML-related overflows would be. Of course, that would cause the second issue for XML. But we could fix that in the same way.  Technically we could do nothing in unmarkgray if we've already tripped the overflow. Though if we're tripping that often enough for this to be a speed win, we've already lost.
I'll assign this to myself for now while I ponder ungraying of shapes.
Assignee: nobody → continuation
Both issues have now been addressed.
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Depends on: 660778
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.