1055163 - Seemingly infinite loop in UnmarkGrayChildren while doing Google signin verification

Status: RESOLVED FIXED

(Core :: JavaScript: GC, defect)

Description

[Benjamin Smedberg]

I was signing into a little-used Google account and it showed me a verification screen asking to call me with a verification code. I entered my phone number and clicked "submit" and the browser hung at 100% CPU.

That was Windows: I tried the same thing on Linux and got the same hang.

Attaching in a debugger shows >	mozjs.dll!MarkInternal<js::Shape>(JSTracer * trc, js::Shape * * thingp) Line 302	C++
 	mozjs.dll!js::gc::MarkShapeUnbarriered(JSTracer * trc, js::Shape * * thingp, const char * name) Line 318	C++
 	mozjs.dll!js::ObjectImpl::markChildren(JSTracer * trc) Line 290	C++
 	mozjs.dll!js::TraceChildren(JSTracer * trc, void * thing, JSGCTraceKind kind) Line 1835	C++
 	mozjs.dll!UnmarkGrayChildren(JSTracer * trc, void * * thingp, JSGCTraceKind kind) Line 1941	C++
 	mozjs.dll!MarkInternal<JSObject>(JSTracer * trc, JSObject * * thingp) Line 298	C++
 	mozjs.dll!js::gc::MarkKind(JSTracer * trc, void * * thingp, JSGCTraceKind kind) Line 633	C++
 	mozjs.dll!MarkValueInternal(JSTracer * trc, JS::Value * v) Line 757	C++
 	mozjs.dll!js::gc::MarkObjectSlots(JSTracer * trc, JSObject * obj, unsigned int start, unsigned int nslots) Line 895	C++
 	mozjs.dll!js::ObjectImpl::markChildren(JSTracer * trc) Line 297	C++
 	mozjs.dll!js::TraceChildren(JSTracer * trc, void * thing, JSGCTraceKind kind) Line 1835	C++
 	mozjs.dll!UnmarkGrayChildren(JSTracer * trc, void * * thingp, JSGCTraceKind kind) Line 1941	C++
 	mozjs.dll!MarkInternal<JSObject>(JSTracer * trc, JSObject * * thingp) Line 298	C++

Which continues as far down as visual studio will show me the stack.

I have this in a debugger and I could save a full memory dump if that would help. Release buildid: 20140815030202

I don't know for certain that the browser is hung, but it stuck for at least 5 minutes like this.

Comment 1

[Terrence Cole [:terrence]]

John also hit this and was kind enough to let me borrow his tmux session for the day. The problem here is that we have a cycle of 2 or more objects that are all in the nursery.  Since they're in the nursery, they're defacto black, but we still have to visit their children to make sure any transitively reachable non-nursery objects are also marked black. This is just missing adequate recursion protection and we didn't catch it until now because it's extremely rare in practice. Patch coming up soon.

Comment 2

[Terrence Cole [:terrence]]

Attachment: fix_unmark_gray_children_iloop-v0.diff

https://tbpl.mozilla.org/?tree=Try&rev=b4c5d5533b1b

Actually, on further thought, if the objects in the nursery are black, they should not contain black->gray edges. This adds a tracer for the UnmarkGray(nurseryObject) case that asserts none of the nursery object's children are gray.

Comment 3

[Terrence Cole [:terrence]]

https://tbpl.mozilla.org/?tree=Try&rev=69e2db966301

And a try push that actually, ah hem, triggers tests.

Comment 4

[Ryan VanderMeulen [:RyanVM]]

https://wiki.mozilla.org/ReleaseEngineering/How_To/Trigger_arbitrary_jobs ;)

Comment 5

[Terrence Cole [:terrence]]

Comment on attachment 8479432 [details] [diff] [review]
fix_unmark_gray_children_iloop-v0.diff

Review of attachment 8479432 [details] [diff] [review]:
-----------------------------------------------------------------

Try run is green.

Comment 6

[Steve Fink [:sfink] [:s:]]

Comment on attachment 8479432 [details] [diff] [review]
fix_unmark_gray_children_iloop-v0.diff

Review of attachment 8479432 [details] [diff] [review]:
-----------------------------------------------------------------

Now that I understand that JS_TraceChildren does nothing of the sort, r=me.

Comment 7

[Terrence Cole [:terrence]]

https://tbpl.mozilla.org/?tree=Try&rev=69e2db966301
https://hg.mozilla.org/integration/mozilla-inbound/rev/f5d3c0ab2837

Comment 9

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/f5d3c0ab2837