781645 - hangs in JS engine with cross fuzzer

Status: RESOLVED WORKSFORME

(Core :: JavaScript Engine, defect)

Description

[Chris Peterson [:cpeterson]]

This bug was filed from the Socorro interface and is 
report bp-810d1dc2-96f0-4adc-822b-9cf192120809 .
============================================================= 

STR:
1. Run "cross_fuzz" browser stress test:
http://lcamtuf.coredump.cx/cross_fuzz/cross_fuzz_msie_randomized_seed.html
2. Wait for a few minutes

AR:
Crash!

I believe this is a recent regression.

I have reproduced this crash on Nightly 8-08 and 8-09 on OS X 10.7.4 (Lion) and Nightly 8-09 on OS X 10.8.0 (ML).

On OS X 10.7, the browser crashed after about 1-10 minutes. On OS X 10.8, the browser crashed in 0-4 minutes, but usually in less than 1 minute.

Comment 1

[Chris Peterson [:cpeterson]]

I also reproduced with Nightly 8-07 on OS X 10.8. I tested earlier builds on OS X 10.7, but I could not repro the crash (but maybe I didn't wait long enough).

Assuming Nightly 8-07 was the regression point, here is the pushlog from 8-06 to 8-07:

https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=c8d94fe7506a&tochange=1bbc0b65dffb

Comment 2

[Andrew McCreight [:mccr8]]

Do you have a crash stack?

Comment 3

[Chris Peterson [:cpeterson]]

The stack trace from the linked crash reports is:

Frame   Module  Signature       Source
0       XUL     XPCWrappedNative::FindTearOff           xpcprivate.h:2631
1       XUL     XPCNativeInterface::GetNewOrUsed        ReentrantMonitor.h:80
2       XUL     nsThreadManager::GetIsMainThread        nsThreadManager.cpp:272
3       XUL     XPCConvert::NativeInterface2JSObject    XPCConvert.cpp:940
4       XUL     js::AutoCompartment::enter              jswrapper.cpp:445
5       XUL     NativeInterface2JSObject                nsXPConnect.cpp:1237
6       XUL     XUL@0x9dc31f    
7       XUL     nsXPConnect::WrapNativeToJSVal          nsXPConnect.cpp:1293
8       XUL     nsCSSStyleDeclSH::PreCreate     
9       XUL     XPCCallContext::XPCCallContext          XPCCallContext.cpp:31
10      XUL     xpc::WrapperFactory::PrepareForWrapping WrapperFactory.cpp:245 

bp-810d1dc2-96f0-4adc-822b-9cf192120809
bp-20e7626a-f7bf-4008-968c-cc5972120809
bp-ef5d2451-2be1-4570-bc59-0faa42120809
bp-e9767b09-9362-4016-a631-6eec82120809
bp-646cd035-17ac-4277-8154-ad1172120809
bp-f54ec5f5-bf76-45f3-9337-0d6922120809
bp-148b49b0-c76d-4297-9e80-38e222120809
bp-e54fa972-c167-4e4a-81a2-9e84f2120809
bp-97237563-1a28-4975-ae20-9345a2120809
bp-32b254aa-2aed-4880-b40f-a82532120809

Comment 4

[Andrew McCreight [:mccr8]]

Attachment: more complete stack from GDB

I was able to reproduce this fairly quickly on OSX 10.6, in GDB. The stack trace looks slightly different for me.

It looks like we're calling XPCWrappedNative::FindTearOff where |this| is NULL.

#0 XPCWrappedNative::FindTearOff js/xpconnect/src/XPCWrappedNative.cpp:1956
#1 XPCConvert::NativeInterface2JSObject js/xpconnect/src/XPCConvert.cpp:940
#2 NativeInterface2JSObject js/xpconnect/src/nsXPConnect.cpp:1237
#3 nsXPConnect::WrapNativeToJSVal js/xpconnect/src/nsXPConnect.cpp:1294
#4 xpc::WrapperFactory::PrepareForWrapping js/xpconnect/wrappers/WrapperFactory.cpp:245
#5 JSCompartment::wrap js/src/jscompartment.cpp:215
#6 JSCompartment::wrap js/src/jscompartment.cpp:327
#7 js::RemapWrapper (cx=0x12167db20, wobj=0x12ff96e00, newTarget=0x14b68c490) at /Users/amccreight/mz/cent2/js/src/jswrapper.cpp:1147
#8 js::RemapAllWrappersForObject js/src/jswrapper.cpp:1189
#9 JS_TransplantObject js/src/jsapi.cpp:1658
#10 xpc::TransplantObject js/xpconnect/wrappers/WrapperFactory.cpp:663
#11 XPCWrappedNative::ReparentWrapperIfFound js/xpconnect/src/XPCWrappedNative.cpp:1682
#12 XPCWrappedNative::RescueOrphans js/xpconnect/src/XPCWrappedNative.cpp:1795
#13 MoveWrapper js/xpconnect/src/nsXPConnect.cpp:1569
#14 nsXPConnect::MoveWrappers js/xpconnect/src/nsXPConnect.cpp:1693
#15 nsHTMLDocument::Open content/html/document/src/nsHTMLDocument.cpp:1503
#16 nsHTMLDocument::WriteCommon content/html/document/src/nsHTMLDocument.cpp:1687
#17 nsIDOMHTMLDocument_Write js/xpconnect/src/dom_quickstubs.cpp:13714

Comment 5

[Boris Zbarsky [:bzbarsky]]

Bobby, this looks like your sort of thing.

Comment 6

[Bobby Holley (:bholley)]

(In reply to Andrew McCreight [:mccr8] from comment #4)
> Created attachment 650709 [details]
> more complete stack from GDB
> 
> I was able to reproduce this fairly quickly on OSX 10.6, in GDB. The stack
> trace looks slightly different for me.
> 
> It looks like we're calling XPCWrappedNative::FindTearOff where |this| is
> NULL.

Indeed. From the previous stack frame, this means that we're pulling the private off an XPCWN reflector when the private is null. Andrew and I have been running into a couple of such cases lately.

So anyway. |flat| (the reflector) appears to have been pulled off the wrapper cache of the underlying XPCOM object. This makes sense, because it looks like we don't update the wrapper cache until after the call to JS_TransplantObject.

You'd think that we'd hit this problem all over the place, because we call JS_WrapObject() in the target object all over the place during transplant. The reason we don't is that normally an object with a wrapper cache also implements PreCreate and requests one reflector per native (rather than one reflector per scope). And the call to WrapNativeToJSVal only happens if we go with one reflector per scope.

It's possible we're getting caught in all that funny business with PreCreate I had to add in PrepareForWrapping (see the mega comment). Either way, the next step is to figure out what kind of object we're wrapping here, and why it's being so funny about PreCreate. Andrew, do you want to investigate that? If you prefer, I can take this over solo, but co-op is always more fun. :-)

Comment 7

[Andrew McCreight [:mccr8]]

I'll try poking at this.

Comment 8

[Andrew McCreight [:mccr8]]

Well, it looks like the native involved is a DOMCSSDeclarationImpl.  I haven't figured out anything else yet.

Comment 9

[Andrew McCreight [:mccr8]]

I tried this again, and I just end up with various hangs in the JS engine. One in the GC, some inside the method JIT. The hangs were slightly odd in that the browser was definitely doing something (breaking again later would end up with a different stack), but I'm not sure what. Anyways, the original XPConnect problem seems to be gone, so I'm going to unassign myself here.

Comment 10

[Naveed Ihsanullah [:naveed]]

I attempted to replicate a JS hang with Nightly 2012-12-02 on OSX ML by going to that URL. I got occasional "Install Error" "This search engine isn't supported by Nightly and can't be installed" but no hang in the browser. 

Is anyone able to replicate the JS in Comment 9?

Comment 11

[Chris Peterson [:cpeterson]]

Naveed, I have not been able to reproduce this bug for a few weeks.

Comment 12

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'XPCWrappedNative::FindTearOff':
 - nightly(version 50):0 crashes from 2016-06-06.
 - aurora (version 49):2 crashes from 2016-06-07.
 - beta   (version 48):26 crashes from 2016-06-06.
 - release(version 47):70 crashes from 2016-05-31.
 - esr    (version 45):4 crashes from 2016-04-07.

Crash volume on the last weeks:
            W. N-1  W. N-2  W. N-3  W. N-4  W. N-5  W. N-6  W. N-7
 - nightly       0       0       0       0       0       0       0
 - aurora        0       0       0       0       1       1       0
 - beta          3       2       5       3       4       8       1
 - release      12       7      13      10       8       8       7
 - esr           1       1       0       0       0       0       0

Affected platforms: Windows, Mac OS X, Linux

Comment 13

[Andrei Purice]

Closing this issue as Resolved > Worksforme since no crashes with this Signature were reported in the last 6 months.